Data source

Questions about the operation or expectations of the NetStumbler software

Data source

Postby Oralis » Thu May 30, 2002 4:35 am

*errrggg*

Not having much luck with figuring out the NS data format. Maybe there could be an ODBC created for it. In short this would give us access to the data without giving away the format secrets.


---Oralis
Oralis
Mini Stumbler
 
Posts: 25
Joined: Tue May 21, 2002 8:40 am
Location: Kansas

Postby 138 » Thu May 30, 2002 5:59 am

Here are a few (not a full list of) hints:
1) Look at the file in a Hex Editor.
2) The SSIDs and Names are obvious, but notice the value immediately preceding them.
3) The MAC address is obvious.
4) Some values are derived, they aren't obvious.
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby Oralis » Thu May 30, 2002 6:09 am

I did look at in both binary and hex editors ..
..and like you saw the ssid's and some other usefull info, but couldnt make heads or tails of the other particulars ..the other issue is, is the file locked and if we try to append data to the file will it fail to open in NS.


---Oralis
Oralis
Mini Stumbler
 
Posts: 25
Joined: Tue May 21, 2002 8:40 am
Location: Kansas

Postby 138 » Thu May 30, 2002 6:26 am

That's because you haven't done enough programming to realize that not every file is delimited, fixed-size, or written with ASCII characters.

You can append data to the file if you know where to update the record count in the file.

More hints:
1) The file keeps track of how many records are in it.
2) How well do you know your binary representations of negative numbers?
3) Do you know the difference between Big and Little Endians?
4) The information to construct the graphs are contained in each record.
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby blackwave » Thu May 30, 2002 6:42 am

Originally posted by 138
More hints:
1) The file keeps track of how many records are in it.
2) How well do you know your binary representations of negative numbers?
3) Do you know the difference between Big and Little Endians?
4) The information to construct the graphs are contained in each record.


Thanks for all your help and hints 138, the few clock ticks we get to research this makes this project move at a snails pace.
-=BW=-
User avatar
blackwave
 
Posts: 4507
Joined: Mon Apr 15, 2002 3:00 am
Location: SoCal, OC

Postby 138 » Thu May 30, 2002 6:50 am

I have about 80% of the format figured out, but I need more data files to finish. Need files with GPS coordinates. Need file with more than 256 records. Need to get off my lazy ass and figure out which format timestamps are stored in.
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby Oralis » Thu May 30, 2002 6:52 am

I do admit to not being a "Programming" so this is all new to me.
Oralis
Mini Stumbler
 
Posts: 25
Joined: Tue May 21, 2002 8:40 am
Location: Kansas

Postby blackwave » Thu May 30, 2002 6:56 am

Originally posted by 138
I have about 80% of the format figured out, but I need more data files to finish. Need files with GPS coordinates. Need file with more than 256 records. Need to get off my lazy ass and figure out which format timestamps are stored in.


138 FYI:
Trading NS1 Files
http://forums.netstumbler.com/showthread.php?s=&threadid=1783

the latest NS1 we have is ~30MB Nationwide (US).. in Monday's update it will be larger still.
-=BW=-
User avatar
blackwave
 
Posts: 4507
Joined: Mon Apr 15, 2002 3:00 am
Location: SoCal, OC

Postby blackwave » Thu May 30, 2002 6:57 am

Originally posted by Oralis
I do admit to not being a "Programming" so this is all new to me.


Don't worry Oralis, this is a learning process for everyone :)
-=BW=-
User avatar
blackwave
 
Posts: 4507
Joined: Mon Apr 15, 2002 3:00 am
Location: SoCal, OC

Postby acidrayne » Thu May 30, 2002 7:01 am

Originally posted by blackwave


138 FYI:
[B]Trading NS1 Files

http://forums.netstumbler.com/showthread.php?s=&threadid=1783

the latest NS1 we have is ~30MB Nationwide (US).. in Monday's update it will be larger still. [/B]


I believe this may have been stated but I cant remember for sure ... Were you also interested in files that do not have GPS??
Rayne
User avatar
acidrayne
 
Posts: 476
Joined: Wed May 01, 2002 7:04 am
Location: Kansas

Postby blackwave » Thu May 30, 2002 7:06 am

Originally posted by acidrayne


I believe this may have been stated but I cant remember for sure ... Were you also interested in files that do not have GPS??


In short yes.

Once we have found out how to take this file apart and repack we plan to separate the gps and non gps data and put it in an alternate database for further analysis that is non-gps based.

It is good to analyze all the other data as well. All information is good. In the current master list we currently have 4577 logged APs with GPS and around 500 without GPS info.
-=BW=-
User avatar
blackwave
 
Posts: 4507
Joined: Mon Apr 15, 2002 3:00 am
Location: SoCal, OC

Postby acidrayne » Thu May 30, 2002 7:10 am

I have a master file that currently has 205 and will keep growing ... would you prefer I just keep sending you updates of the master as I get them?
Rayne
User avatar
acidrayne
 
Posts: 476
Joined: Wed May 01, 2002 7:04 am
Location: Kansas

Postby 138 » Thu May 30, 2002 7:12 am

A record appears to start with the length of the string containing the SSID, followed by the SSID, and MAC address. Next appears to be where NS stores the information that is used to construct the graphs. (Guessing) Sig+, Noise-, stuff I haven't *quite* figured out yet. These sub-records are repeated for each spike on the graph. The last information stored in the record is the length of the string representing the Name of the AP, followed by the Name of the AP represented as a String (if applicable).

IE - If the AP doesn't have a name, the record ends with 16 zeroes.
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby blackwave » Thu May 30, 2002 7:16 am

Originally posted by acidrayne
I have a master file that currently has 205 and will keep growing ... would you prefer I just keep sending you updates of the master as I get them?


That would be wonderful thanks! :)
-=BW=-
User avatar
blackwave
 
Posts: 4507
Joined: Mon Apr 15, 2002 3:00 am
Location: SoCal, OC

Postby acidrayne » Thu May 30, 2002 7:20 am

Originally posted by blackwave


That would be wonderful thanks! :)


Will probably get that to you tonight after work ... Maybe on lunch if I have time.
Rayne
User avatar
acidrayne
 
Posts: 476
Joined: Wed May 01, 2002 7:04 am
Location: Kansas

Next

Return to Help

Who is online

Users browsing this forum: No registered users and 2 guests

cron