NetStumbler.org Forums

*errrggg*

Not having much luck with figuring out the NS data format. Maybe there could be an ODBC created for it. In short this would give us access to the data without giving away the format secrets.


---Oralis

Here are a few (not a full list of) hints:
1) Look at the file in a Hex Editor.
2) The SSIDs and Names are obvious, but notice the value immediately preceding them.
3) The MAC address is obvious.
4) Some values are derived, they aren't obvious.

I did look at in both binary and hex editors ..
..and like you saw the ssid's and some other usefull info, but couldnt make heads or tails of the other particulars ..the other issue is, is the file locked and if we try to append data to the file will it fail to open in NS.


---Oralis

That's because you haven't done enough programming to realize that not every file is delimited, fixed-size, or written with ASCII characters.

You can append data to the file if you know where to update the record count in the file.

More hints:
1) The file keeps track of how many records are in it.
2) How well do you know your binary representations of negative numbers?
3) Do you know the difference between Big and Little Endians?
4) The information to construct the graphs are contained in each record.

Originally posted by 138
More hints:
1) The file keeps track of how many records are in it.
2) How well do you know your binary representations of negative numbers?
3) Do you know the difference between Big and Little Endians?
4) The information to construct the graphs are contained in each record.

Thanks for all your help and hints 138, the few clock ticks we get to research this makes this project move at a snails pace.

I have about 80% of the format figured out, but I need more data files to finish. Need files with GPS coordinates. Need file with more than 256 records. Need to get off my lazy ass and figure out which format timestamps are stored in.

I do admit to not being a "Programming" so this is all new to me.

Originally posted by 138
I have about 80% of the format figured out, but I need more data files to finish. Need files with GPS coordinates. Need file with more than 256 records. Need to get off my lazy ass and figure out which format timestamps are stored in.

138 FYI:
Trading NS1 Files
http://forums.netstumbler.com/showthread.php?s=&threadid=1783

the latest NS1 we have is ~30MB Nationwide (US).. in Monday's update it will be larger still.

Originally posted by Oralis
I do admit to not being a "Programming" so this is all new to me.

Don't worry Oralis, this is a learning process for everyone

Originally posted by blackwave


138 FYI:
[B]Trading NS1 Files
http://forums.netstumbler.com/showthread.php?s=&threadid=1783

the latest NS1 we have is ~30MB Nationwide (US).. in Monday's update it will be larger still. [/B]

I believe this may have been stated but I cant remember for sure ... Were you also interested in files that do not have GPS??

Originally posted by acidrayne


I believe this may have been stated but I cant remember for sure ... Were you also interested in files that do not have GPS??

In short yes.

Once we have found out how to take this file apart and repack we plan to separate the gps and non gps data and put it in an alternate database for further analysis that is non-gps based.

It is good to analyze all the other data as well. All information is good. In the current master list we currently have 4577 logged APs with GPS and around 500 without GPS info.

I have a master file that currently has 205 and will keep growing ... would you prefer I just keep sending you updates of the master as I get them?

A record appears to start with the length of the string containing the SSID, followed by the SSID, and MAC address. Next appears to be where NS stores the information that is used to construct the graphs. (Guessing) Sig+, Noise-, stuff I haven't *quite* figured out yet. These sub-records are repeated for each spike on the graph. The last information stored in the record is the length of the string representing the Name of the AP, followed by the Name of the AP represented as a String (if applicable).

IE - If the AP doesn't have a name, the record ends with 16 zeroes.

Originally posted by acidrayne
I have a master file that currently has 205 and will keep growing ... would you prefer I just keep sending you updates of the master as I get them?

That would be wonderful thanks!

Originally posted by blackwave


That would be wonderful thanks!

Will probably get that to you tonight after work ... Maybe on lunch if I have time.