Data source

Questions about the operation or expectations of the NetStumbler software

Postby 138 » Thu May 30, 2002 10:45 am

Originally posted by lincomatic


the integers are reversed because Wintel uses little endian storage order. and same on the pocketpc's...the ARM can be configured for either little or big endian, but MS uses it in little endian mode, so the binary files are compatible w/ the desktops.


Thank you, that is what I was thinking. Of course finding the dates sucks, I don't remember my binary date formats that well. I am assuming it is represented in milliseconds from 1/1/1970, and that it is in little endian form, but I'm unsure how many bits the file format is using to represent the time/date. Any suggestions?
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby lincomatic » Thu May 30, 2002 11:19 am

Originally posted by 138


Thank you, that is what I was thinking. Of course finding the dates sucks, I don't remember my binary date formats that well. I am assuming it is represented in milliseconds from 1/1/1970, and that it is in little endian form, but I'm unsure how many bits the file format is using to represent the time/date. Any suggestions?


my guess is it's using the format from the time() function so it should be a 32 bit integer.

here is what i have so far starting from the beginning of a file:
4E 65 74 53 -> NetS -> magic string
06 00 00 00 -> 6???
76 01 00 00 -> 176H -> 374 -> total AP count in file
04 -> SSID string length
41 44 41 4D -> ADAM -> SSID
00 04 5A 0E 32 89 -> AP MAC address
A9 FF FF FF -> -87 -> signal
97 FF FF FF -> -105 -> noise
0E 00 00 00 -> 14 -> SNR
08 00 00 00 -> ??
01 00 00 00 -> 0001 -> flags
64 00 00 00 -> 100 -> beacon
3F D1 94 1E -> i'm lost from here on out
38 C1 01 80
3E A5 F9 8B

interestingly, if you sit still and only have 1 AP the file keeps growing because it actually logs all the encountered signal strengths at the scan interval so you have a history of it.
~lincomatic
User avatar
lincomatic
Mini Stumbler
 
Posts: 1682
Joined: Tue Apr 16, 2002 12:53 am
Location: Tinsel Town

Postby 138 » Thu May 30, 2002 11:52 am

Originally posted by lincomatic


my guess is it's using the format from the time() function so it should be a 32 bit integer.

here is what i have so far starting from the beginning of a file:
4E 65 74 53 -> NetS -> magic string
06 00 00 00 -> 6???
76 01 00 00 -> 176H -> 374 -> total AP count in file
04 -> SSID string length
41 44 41 4D -> ADAM -> SSID
00 04 5A 0E 32 89 -> AP MAC address
A9 FF FF FF -> -87 -> signal
97 FF FF FF -> -105 -> noise
0E 00 00 00 -> 14 -> SNR
08 00 00 00 -> ??
01 00 00 00 -> 0001 -> flags
64 00 00 00 -> 100 -> beacon
3F D1 94 1E -> i'm lost from here on out
38 C1 01 80
3E A5 F9 8B

interestingly, if you sit still and only have 1 AP the file keeps growing because it actually logs all the encountered signal strengths at the scan interval so you have a history of it.


Okay, I haven't closely checked my notes, but I believe you agree with what I see. I think the 08 00 00 00 is the channel. I noticed that when you export a file to text, it doesn't store the channel, but the "channelbits". Try exporting your file and see if it matches. My theory is that the channelbits determines the channel, but I haven't had time to look into furthur.

Okay, where you say "i'm lost form here on out", I am working on the theory that the next part is the time/date. I don't have files that have lat/long, but I think that is after time/date. I don't have a clue right now on the rest of it.

You will notice that the signal strengths will repeat until the very end of the record where there will be something like:

00 -> length of AP Name (if there isn't one)
-or-
07 -> lenght of AP Name (for this example "linksys")
6C 69 6E 6B 73 79 73 -> AP Name "linksys"
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby 138 » Thu May 30, 2002 11:54 am

Originally posted by lincomatic


my guess is it's using the format from the time() function so it should be a 32 bit integer.

here is what i have so far starting from the beginning of a file:
4E 65 74 53 -> NetS -> magic string
06 00 00 00 -> 6???
76 01 00 00 -> 176H -> 374 -> total AP count in file
04 -> SSID string length
41 44 41 4D -> ADAM -> SSID
00 04 5A 0E 32 89 -> AP MAC address
A9 FF FF FF -> -87 -> signal
97 FF FF FF -> -105 -> noise
0E 00 00 00 -> 14 -> SNR
08 00 00 00 -> ??
01 00 00 00 -> 0001 -> flags
64 00 00 00 -> 100 -> beacon
3F D1 94 1E -> i'm lost from here on out
38 C1 01 80
3E A5 F9 8B

interestingly, if you sit still and only have 1 AP the file keeps growing because it actually logs all the encountered signal strengths at the scan interval so you have a history of it.


Oh yeah, I am probably stating the obvious, but just in case, the manufacturer is being determined from the MAC address.
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby lincomatic » Thu May 30, 2002 1:09 pm

brilliant deduction - that really is an obvious way to get the vendor...from the mac...i hadn't thought of that.

looks like we are definitely on the same track...i noticed that he put the AP name at the end, too...weird.

tried putting those weird #'s into ctime() but they just gave me gibberish instead of valid dates. i wonder if he's using windows time formats instead.

you're right about the channel bits. here is what i got when i switched channels:

channel 1 -> 02 00 00 00
channel 2 -> 04 00 00 00
channel 3 -> 08 00 00 00
channel 3,6>48 00 00 00
channel 1,6>42 00 00 00


so bit1=1, bit2=2, etc.

also the flag bits in 0011 <- the first 1 is WEP enabled.
w/ WEP disabled, most have 0001. don't know what 0005 means, though...weird value i got on one.
~lincomatic
User avatar
lincomatic
Mini Stumbler
 
Posts: 1682
Joined: Tue Apr 16, 2002 12:53 am
Location: Tinsel Town

Postby lincomatic » Thu May 30, 2002 1:50 pm

yah know, i just started looking at the text export format to see the "channelbits" field you mentioned, and noticed...it has all the info that the NS1 file has, doesn't it? and NS can read those text files back in and display everything you see when it reads .NS1 files....

which brings to mind the question: why are we even bothering to figure out the binary format???
~lincomatic
User avatar
lincomatic
Mini Stumbler
 
Posts: 1682
Joined: Tue Apr 16, 2002 12:53 am
Location: Tinsel Town

Postby blackwave » Thu May 30, 2002 2:01 pm

Originally posted by lincomatic
yah know, i just started looking at the text export format to see the "channelbits" field you mentioned, and noticed...it has all the info that the NS1 file has, doesn't it? and NS can read those text files back in and display everything you see when it reads .NS1 files....

which brings to mind the question: why are we even bothering to figure out the binary format???


Which begs the question.. why is there a Binary format?

Personally I wanted to rip and repack ns1 since this is the current convention for ns file trading..
-=BW=-
User avatar
blackwave
 
Posts: 4507
Joined: Mon Apr 15, 2002 3:00 am
Location: SoCal, OC

Postby lincomatic » Thu May 30, 2002 2:08 pm

bw,
so you can do it by
1) exporting to text and doing whatever you want to them.
2) reading the files back into NS1
3) writing them out as NS1 again.

is this good enough or do you have something fancier in mind?
~lincomatic
User avatar
lincomatic
Mini Stumbler
 
Posts: 1682
Joined: Tue Apr 16, 2002 12:53 am
Location: Tinsel Town

Postby blackwave » Thu May 30, 2002 2:11 pm

Originally posted by lincomatic
bw,
so you can do it by
1) exporting to text and doing whatever you want to them.
2) reading the files back into NS1
3) writing them out as NS1 again.

is this good enough or do you have something fancier in mind?


<looking around> Netstumbler supports converting a Summary/Text file into an NS1 binary file?...

What would be nice would be an api or something because this automation is all through scripting.. I would love to have more automated control.
-=BW=-
User avatar
blackwave
 
Posts: 4507
Joined: Mon Apr 15, 2002 3:00 am
Location: SoCal, OC

Postby lincomatic » Thu May 30, 2002 2:15 pm

yeah an external api spec for controlling its inner workings would be nice...

try this:
1) export to text
2) drag the text file over NS or use the open command
3) save

you now have an NS1 file from a text file.
~lincomatic
User avatar
lincomatic
Mini Stumbler
 
Posts: 1682
Joined: Tue Apr 16, 2002 12:53 am
Location: Tinsel Town

Postby lincomatic » Thu May 30, 2002 2:23 pm

actually i can see why u want to automate things...
but what i am afraid of is that marius will change the binary format again and then we'll get stuck decoding it all over.
text files definitely take more disk space too.

perhaps we should just use zipped text as the upload format and then you can just convert them for your database?
~lincomatic
User avatar
lincomatic
Mini Stumbler
 
Posts: 1682
Joined: Tue Apr 16, 2002 12:53 am
Location: Tinsel Town

Postby blackwave » Thu May 30, 2002 2:23 pm

Originally posted by lincomatic
yeah an external api spec for controlling its inner workings would be nice...

try this:
1) export to text
2) drag the text file over NS or use the open command
3) save

you now have an NS1 file from a text file.


Thanks lincomatic, I will have to try this when I get home! If this works perhaps I can use visual script to click buttons for me after a script operation is complete... :) Though I really would like to create my own ns1's.. but I may have to pursue that alone. :)
-=BW=-
User avatar
blackwave
 
Posts: 4507
Joined: Mon Apr 15, 2002 3:00 am
Location: SoCal, OC

Postby 138 » Thu May 30, 2002 2:53 pm

Originally posted by lincomatic
brilliant deduction - that really is an obvious way to get the vendor...from the mac...i hadn't thought of that.

looks like we are definitely on the same track...i noticed that he put the AP name at the end, too...weird.

tried putting those weird #'s into ctime() but they just gave me gibberish instead of valid dates. i wonder if he's using windows time formats instead.

you're right about the channel bits. here is what i got when i switched channels:

channel 1 -> 02 00 00 00
channel 2 -> 04 00 00 00
channel 3 -> 08 00 00 00
channel 3,6>48 00 00 00
channel 1,6>42 00 00 00


so bit1=1, bit2=2, etc.

also the flag bits in 0011 <- the first 1 is WEP enabled.
w/ WEP disabled, most have 0001. don't know what 0005 means, though...weird value i got on one.


Cool...I hadn't even started looking for the WEP bit yet.

I don't think the date is in Windows format. I played with it for a little while but got back some really funky stuff (MS time starts on 1899.12.31 BTW).
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby 138 » Thu May 30, 2002 2:55 pm

Originally posted by lincomatic
yah know, i just started looking at the text export format to see the "channelbits" field you mentioned, and noticed...it has all the info that the NS1 file has, doesn't it? and NS can read those text files back in and display everything you see when it reads .NS1 files....

which brings to mind the question: why are we even bothering to figure out the binary format???



Umm...Ummm...because its more manly? Because we are proving our intellectual worth/problem solving skills?


Ya know, I just assumed they couldn't be imported and/or merged. I didn't think anyone would have asked about the binary format unless the text version didn't work.
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

Postby 138 » Thu May 30, 2002 3:00 pm

Originally posted by lincomatic
actually i can see why u want to automate things...
but what i am afraid of is that marius will change the binary format again and then we'll get stuck decoding it all over.
text files definitely take more disk space too.

perhaps we should just use zipped text as the upload format and then you can just convert them for your database?



AND, the text file is tab delimited (Oralis). Funny though, I don't see the date anywhere in the text file. Time, yes. Date, no.

Besides, at least now we know what stuff like channelbits are. So we would have had to hack the binary anyway.
138
Mini Stumbler
 
Posts: 17
Joined: Thu May 30, 2002 5:53 am

PreviousNext

Return to Help

Who is online

Users browsing this forum: No registered users and 2 guests

cron