Error when downloading pcap's from the Nano

[crashie]

Hi, I've run into another issue with the NANO. When trying to download the pcap captures from the NANO either from the webinterface or using sftp it terminates the transfer after 5-10 seconds. I have a pcap of around 130Mb that I want to download to my laptop but I'll probably have to take out the SD-card and put it into another device and extract the pcap's.

Anyone else that have had this issue?

/Crashie

[dustbyter]

I have not seen any issues with using SFTP to transfer files, but I have to say the files were much smaller, on the order of 10-20 Mb.

Does the device hang, or just the transfer is terminated?

[crashie]

I have not seen any issues with using SFTP to transfer files, but I have to say the files were much smaller, on the order of 10-20 Mb.

Does the device hang, or just the transfer is terminated?

Correction, I've now seen that the Nano actually disconnects and reconnects when looking at it in the network settings page on my mac. And therefore the transmission gets interrupted. Don't really understand why though.. somethings weird.

I've also done a "factory reset" of the Nano but it still gives me a hard time trying to download files from it. Whether it's connected using USB, or when over Wi-Fi. As soon as I start downloading it disconnects after some seconds. It's like the network interface on it dies for just 2 seconds and then it comes up, but it's enough to interrupt the transmission.

I've also noticed that if a client is connected and starts a heavy download it behaves the same way, the network interface disconnects and reconnects. Think I'll have to reset again and try once more.. but something's wrong..

Edited February 16, 2016 by crashie

[crashie]

Correction, I've now seen that the Nano actually disconnects and reconnects when looking at it in the network settings page on my mac. And therefore the transmission gets interrupted. Don't really understand why though.. somethings weird.

I've also done a "factory reset" of the Nano but it still gives me a hard time trying to download files from it. Whether it's connected using USB, or when over Wi-Fi. As soon as I start downloading it disconnects after some seconds. It's like the network interface on it dies for just 2 seconds and then it comes up, but it's enough to interrupt the transmission.

I've also noticed that if a client is connected and starts a heavy download it behaves the same way, the network interface disconnects and reconnects. Think I'll have to reset again and try once more.. but something's wrong..

Update.. Did a full reflash of the firmware pushing in the reset button while connecting the nano to my USB. Then flashed the latest firmware again and configured and installed only urlsnarf and tcpdump modules. Now it seems fine and I could download a 138Mb pcap from it without issues. Going to install one module after another and see why it did what it did before.. But that's for tomorrow. Now it's sleeping time :P

[crashie]

Update.. Did a full reflash of the firmware pushing in the reset button while connecting the nano to my USB. Then flashed the latest firmware again and configured and installed only urlsnarf and tcpdump modules. Now it seems fine and I could download a 138Mb pcap from it without issues. Going to install one module after another and see why it did what it did before.. But that's for tomorrow. Now it's sleeping time :P

Ok, another update :P

The issue is back again and I've only installed the urlsnarf and tcpdump modules on the SD-card. As soon as I initiate a download from the NANO it starts to disconnect and reconnect the interface and behaving crazy. Wondering if it's about heating issues or something else. It doesn't matter if I try to do it connected directly to the NANO with through the USB-port or over the Wi-Fi. The same result in the end. Always disconnects after some seconds, reconnects and disconnects and so on for a while and in the end interrupting the transfer. Doesn't matter which computer I try from, it's the same result. Maybe it's my bad luck and the device is faulty? Or there is something I can do to solve it..

Another thing I've noticed today is that when I've "checked" the PineAP "Allow Associations" box and starts it disables itself after a while. Sometimes not at all but sometimes on it's own after just minutes. I recheck it and it stays enabled for a while after that. But somethings really weird, that I can say at least. Hopefully someone here have some ideas to try :)

/crashie

[crashie]

Ok, another update :P

The issue is back again and I've only installed the urlsnarf and tcpdump modules on the SD-card. As soon as I initiate a download from the NANO it starts to disconnect and reconnect the interface and behaving crazy. Wondering if it's about heating issues or something else. It doesn't matter if I try to do it connected directly to the NANO with through the USB-port or over the Wi-Fi. The same result in the end. Always disconnects after some seconds, reconnects and disconnects and so on for a while and in the end interrupting the transfer. Doesn't matter which computer I try from, it's the same result. Maybe it's my bad luck and the device is faulty? Or there is something I can do to solve it..

Another thing I've noticed today is that when I've "checked" the PineAP "Allow Associations" box and starts it disables itself after a while. Sometimes not at all but sometimes on it's own after just minutes. I recheck it and it stays enabled for a while after that. But somethings really weird, that I can say at least. Hopefully someone here have some ideas to try :)

/crashie

Ok, next update on the issue..

It seems to relate to the USB (or microSD) interface.. maybe I've got a defect NANO or SD (tried another SD-card but that doesn't mount)?.. Been trying everything but as soon as I initiate a transfer of a file that is large enough the interface goes down and then back up in 3-5 seconds and using sftp I can resume the download but after 10-40 seconds (differs from time to time) then it disconnects again and reconnects which interrupts the transfer.. So to be able to transfer a pcap of 166Mb I have to resume the download 30-50 times before I've got the whole pcap..

I've recorded a video of the screen of what's going on when it happens.. but below is some data from dmesg:

[ 459.450000] usb 1-1.2: USB disconnect, device number 6

[ 459.460000] scsi 2:0:0:0: rejecting I/O to offline device

[ 459.460000] scsi 2:0:0:0: [sdc] killing request

[ 459.470000] scsi 2:0:0:0: [sdc]

[ 459.470000] Result: hostbyte=0x01 driverbyte=0x00

[ 459.470000] scsi 2:0:0:0: [sdc] CDB:

[ 459.480000] cdb[0]=0x28: 28 00 00 00 00 5f 00 00 40 00

[ 459.480000] blk_update_request: I/O error, dev sdc, sector 95

[ 459.490000] Buffer I/O error on dev sdc2, logical block 28, async page read

[ 459.490000] Buffer I/O error on dev sdc2, logical block 29, async page read

[ 459.500000] Buffer I/O error on dev sdc2, logical block 30, async page read

[ 459.510000] Buffer I/O error on dev sdc2, logical block 31, async page read

[ 460.370000] eth0: link down

[ 460.370000] br-lan: port 1(eth0) entered disabled state

[ 461.480000] device wlan0 entered promiscuous mode

[ 461.480000] br-lan: port 2(wlan0) entered forwarding state

[ 461.490000] br-lan: port 2(wlan0) entered forwarding state

[ 461.610000] device wlan0-1 entered promiscuous mode

[ 461.620000] br-lan: port 3(wlan0-1) entered forwarding state

[ 461.620000] br-lan: port 3(wlan0-1) entered forwarding state

[ 462.600000] usb 1-1.2: new high-speed USB device number 7 using ehci-platform

[ 462.610000] br-lan: port 3(wlan0-1) entered disabled state

[ 462.740000] usb-storage 1-1.2:1.0: USB Mass Storage device detected

[ 462.780000] scsi host3: usb-storage 1-1.2:1.0

[ 463.340000] br-lan: port 3(wlan0-1) entered forwarding state

[ 463.340000] br-lan: port 3(wlan0-1) entered forwarding state

[ 463.490000] br-lan: port 2(wlan0) entered forwarding state

[ 463.610000] eth0: link up (100Mbps/Full duplex)

[ 463.610000] br-lan: port 1(eth0) entered forwarding state

[ 463.620000] br-lan: port 1(eth0) entered forwarding state

[ 463.780000] scsi 3:0:0:0: Direct-Access Generic STORAGE DEVICE 0933 PQ: 0 ANSI: 6

[ 463.790000] sd 3:0:0:0: Attached scsi generic sg0 type 0

[ 464.060000] sd 3:0:0:0: [sdc] 62521344 512-byte logical blocks: (32.0 GB/29.8 GiB)

[ 464.070000] sd 3:0:0:0: [sdc] Write Protect is off

[ 464.070000] sd 3:0:0:0: [sdc] Mode Sense: 21 00 00 00

[ 464.070000] sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA

[ 464.120000] sdc: sdc1 sdc2

[ 464.130000] sd 3:0:0:0: [sdc] Attached SCSI removable disk

[ 465.340000] br-lan: port 3(wlan0-1) entered forwarding state

[ 465.620000] br-lan: port 1(eth0) entered forwarding state

[ 466.340000] Buffer I/O error on dev sdb1, logical block 3702784, lost sync page write

[ 466.350000] JBD2: Error -5 detected when updating journal superblock for sdb1-8.

[ 466.350000] Aborting journal on device sdb1-8.

[ 466.360000] Buffer I/O error on dev sdb1, logical block 3702784, lost sync page write

[ 466.370000] JBD2: Error -5 detected when updating journal superblock for sdb1-8.

[ 466.420000] EXT4-fs (sdc1): couldn't mount as ext3 due to feature incompatibilities

[ 466.460000] EXT4-fs (sdc1): couldn't mount as ext2 due to feature incompatibilities

[ 466.600000] EXT4-fs (sdc1): recovery complete

[ 466.610000] EXT4-fs (sdc1): mounted filesystem with ordered data mode. Opts: (null)

[ 466.980000] Adding 1004024k swap on /dev/sdcard/sd2. Priority:-3 extents:1 across:1004024k

I've also now tried with another SD-card (2GB) but still have the same issue. I can upload a 160MB pcap to the SD storage through SFTP with no errors. But as soon as I try to download the same file it disconnects and reconnects.

With my other SD-card I got this error also if it can help:

[ 471.720000] usb 1-1.2: USB disconnect, device number 11

[ 471.720000] scsi 7:0:0:0: rejecting I/O to offline device

[ 471.720000] scsi 7:0:0:0: [sdb] killing request

[ 471.730000] scsi 7:0:0:0: [sdb]

[ 471.740000] Result: hostbyte=0x01 driverbyte=0x00

[ 471.740000] scsi 7:0:0:0: [sdb] CDB:

[ 471.740000] cdb[0]=0x28: 28 00 00 00 00 00 00 00 20 00

[ 471.750000] blk_update_request: I/O error, dev sdb, sector 0

[ 471.760000] Buffer I/O error on dev sdb, logical block 0, async page read

/crashie

Edited February 17, 2016 by crashie

[crashie]

Some more data:

Wed Feb 17 21:15:42 2016 kern.info kernel: [ 118.760000] br-lan: port 3(wlan0-1) entered disabled state
Wed Feb 17 21:15:42 2016 kern.info kernel: [ 118.760000] device wlan0-1 left promiscuous mode
Wed Feb 17 21:15:42 2016 kern.info kernel: [ 118.770000] br-lan: port 3(wlan0-1) entered disabled state
Wed Feb 17 21:15:42 2016 daemon.notice netifd: Network device 'wlan0-1' link is down
Wed Feb 17 21:15:42 2016 kern.info kernel: [ 118.910000] device wlan0 left promiscuous mode
Wed Feb 17 21:15:42 2016 kern.info kernel: [ 118.910000] br-lan: port 2(wlan0) entered disabled state
Wed Feb 17 21:15:42 2016 daemon.notice netifd: Network device 'wlan0' link is down
Wed Feb 17 21:15:42 2016 kern.err kernel: [ 119.570000] Buffer I/O error on dev sda1, logical block 2655233, lost sync page write
Wed Feb 17 21:15:42 2016 kern.err kernel: [ 119.570000] JBD2: Error -5 detected when updating journal superblock for sda1-8.
Wed Feb 17 21:15:42 2016 kern.err kernel: [ 119.580000] Aborting journal on device sda1-8.
Wed Feb 17 21:15:42 2016 kern.err kernel: [ 119.580000] Buffer I/O error on dev sda1, logical block 2655233, lost sync page write
Wed Feb 17 21:15:42 2016 kern.err kernel: [ 119.590000] JBD2: Error -5 detected when updating journal superblock for sda1-8.
Wed Feb 17 21:15:42 2016 kern.err kernel: [ 119.620000] EXT4-fs (sdb1): couldn't mount as ext3 due to feature incompatibilities
Wed Feb 17 21:15:43 2016 kern.err kernel: [ 119.650000] EXT4-fs (sdb1): couldn't mount as ext2 due to feature incompatibilities
Wed Feb 17 21:15:43 2016 daemon.notice netifd: radio1 (2255): Interface 0 setup failed: BRIDGE_NOT_ALLOWED
Wed Feb 17 21:15:43 2016 daemon.notice netifd: Network device 'eth0' link is down
Wed Feb 17 21:15:43 2016 kern.info kernel: [ 119.970000] eth0: link down
Wed Feb 17 21:15:43 2016 kern.info kernel: [ 119.970000] br-lan: port 1(eth0) entered disabled state
Wed Feb 17 21:15:43 2016 kern.info kernel: [ 120.180000] EXT4-fs (sdb1): recovery complete
Wed Feb 17 21:15:43 2016 kern.info kernel: [ 120.180000] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
Wed Feb 17 21:15:43 2016 kern.info kernel: [ 120.360000] usb 1-1.2: USB disconnect, device number 5
Wed Feb 17 21:15:43 2016 kern.err kernel: [ 120.370000] scsi 1:0:0:0: rejecting I/O to offline device
Wed Feb 17 21:15:43 2016 kern.info kernel: [ 120.370000] scsi 1:0:0:0: [sdb] killing request
Wed Feb 17 21:15:43 2016 kern.info kernel: [ 120.370000] scsi 1:0:0:0: [sdb]
Wed Feb 17 21:15:43 2016 kern.warn kernel: [ 120.380000] Result: hostbyte=0x01 driverbyte=0x00
Wed Feb 17 21:15:43 2016 kern.info kernel: [ 120.380000] scsi 1:0:0:0: [sdb] CDB:
Wed Feb 17 21:15:43 2016 kern.warn kernel: [ 120.390000] cdb[0]=0x28: 28 00 00 00 00 b7 00 00 08 00
Wed Feb 17 21:15:43 2016 kern.err kernel: [ 120.390000] blk_update_request: I/O error, dev sdb, sector 183
Wed Feb 17 21:15:43 2016 kern.err kernel: [ 120.400000] Buffer I/O error on dev sdb2, logical block 60, async page read
Wed Feb 17 21:15:43 2016 kern.err kernel: [ 120.400000] Buffer I/O error on dev sdb2, logical block 61, async page read
Wed Feb 17 21:15:43 2016 kern.err kernel: [ 120.410000] Buffer I/O error on dev sdb2, logical block 62, async page read
Wed Feb 17 21:15:43 2016 kern.err kernel: [ 120.420000] Buffer I/O error on dev sdb2, logical block 63, async page read
Wed Feb 17 21:15:44 2016 kern.info kernel: [ 120.970000] eth0: link up (100Mbps/Full duplex)
Wed Feb 17 21:15:44 2016 kern.info kernel: [ 120.970000] br-lan: port 1(eth0) entered forwarding state

[crashie]

To add to the issue, I've now tried four different microSD-cards. And the same issue with all of them. Are all SD-cards faulty or is it the NANO who's acting up? Can write to the SD on the NANO but the problem starts when reading a large file (downloading) from it through either the webinterface or sftp etc.

[AlfAlfa]

I know sftp is kind of similar to scp but not exactly so what about scp?, did you try scp when you said 'webinterface or sftp etc'? I wonder if that will work:

scp root@172.42.16.1:/root/caps/largecap.pcap ~/caps/largecap.pcap

Excuse my post if you've already tried it, but you weren't clear on that. Maybe when downloading it uses up all or too much memory and that causes something vital to crash creating the issue you find... That's just a guess though because it's hard to see what's going on from your log files (as it looks almost like it was unplugged and re-plugged). Which is the first line that the transfer stops '[ 459.450000] usb 1-1.2: USB disconnect, device number 6' that one?

[crashie]

I know sftp is kind of similar to scp but not exactly so what about scp?, did you try scp when you said 'webinterface or sftp etc'? I wonder if that will work:

scp root@172.42.16.1:/root/caps/largecap.pcap ~/caps/largecap.pcap

Excuse my post if you've already tried it, but you weren't clear on that. Maybe when downloading it uses up all or too much memory and that causes something vital to crash creating the issue you find... That's just a guess though because it's hard to see what's going on from your log files (as it looks almost like it was unplugged and re-plugged). Which is the first line that the transfer stops '[ 459.450000] usb 1-1.2: USB disconnect, device number 6' that one?

The issue is taken care of by sebkinne. He's looking into it since he could also reproduce the same issue. Probably solved soon enough.

[Purrball]

I've had the problem as well and thought I was just doing something incorrectly. After 3 flashes I gave up. It appeared to happen only with urlsnarf and tcpdump running or installed, I narrowed it down to them as well, but figured again... that I just screwed some setting up.

​

​Either way the end result was always a crash or rebooting of the interface... it seemed to me that it was totally rebooting, but wasn't sure. Thank you for posting this, looking forward to a potential fix.

[Sebkinne]

This is a really interesting issue, which I'll hopefully get fixed soon.

I have a few suspicions, so I'm sure it won't take long.

Best Regards,

Sebkinne

[crashie]

This is a really interesting issue, which I'll hopefully get fixed soon.

I have a few suspicions, so I'm sure it won't take long.

Best Regards,

Sebkinne

Yeah, interesting at least :) Looking forward to have it fixed so I can download those big pcaps without a hassle :) I have faith in you.

[Rose Kolodny]

Just encountered this as well. Over the web interface, trying to move a 90mb pcap bricked the device. Moving over scp didn't brick, but failed and threw an IO error. Tried to gzip the file to see if that would help, but the gzip threw a short write error. Tried using tcpdump -C 5 over ssh to have it split the pcap into 5mb pieces, but that threw an IO error. Tried to throttle the scp with -l 1 to limit the speed, but that also threw an IO error.

[Badibou]

Same problem here (and more generally I believe any I/O to/from the SD randomly triggers such faults). Very interested to know if there is a fix.

[crashie]

Lets hope Sebkinne can fix the issue.

Same problem here (and more generally I believe any I/O to/from the SD randomly triggers such faults). Very interested to know if there is a fix.

[Badibou]

Hi,

Do you have any news about this bug please ?

Without reliable SD support the Nano is useless to me...

Thanks

This is a really interesting issue, which I'll hopefully get fixed soon.

I have a few suspicions, so I'm sure it won't take long.

Best Regards,

Sebkinne

[Mr.miYagi]

Hi

 

I have the same issue an my MKV with FW 3.0

Had the same issue with downloading files from the sslsplit modules.

And now i get errors with some modules installed on the sd.

I installed python 2.7 on the SD. When i start SSLstrip, i get a BUS error.

After that i can see in dmesg the I/O error or a bad superblock warnig.

Badblock cant find any bad blocks....

I remove the swap partition and retry the hole story, mabe this helps to reduce the bottleneck or read/write errors. Maybe it helps.

same issues with original sd / samsung 16GB class 10 / Kingston 16GB class 10

 

Edit:

copied all files from SD to PC, formatted the SD, checked for errors. Created only 1 ext4 partition without swap.

Copied the files back, after boot, copied a big folder and i get this:

[  682.750000] blk_update_request: I/O error, dev sdb, sector 397280
[  683.280000] EXT4-fs warning (device sdb1): ext4_end_bio:317: I/O error -5 writing to inode 2841 (offset 0 size 4096 starting block 54878)
[  683.280000] Buffer I/O error on device sdb1, logical block 54621
[  683.290000] Aborting journal on device sdb1-8.
[  683.300000] JBD2: Error -5 detected when updating journal superblock for sdb1-8.
[  683.310000] EXT4-fs error (device sdb1): ext4_journal_check_start:56: Detected aborted journal
[  683.310000] EXT4-fs (sdb1): Remounting filesystem read-only
[  683.320000] EXT4-fs (sdb1): previous I/O error to superblock detected

Edited October 6, 2016 by Mr.miYagi

[Kazansky]

Any update on this ? I have exactly the same problem when transferring pcap files from the SD

to the main host through USB.

Symptoms are : 

1) Transfer with scp starts well ;

2) After a while, got a "stalled" status on scp ;

3) Doing a "ls " on the directory on the SD cards returns empty list (no files) ;

Thanks !! Alain.

[Zylla]

It might seem to be related to a bug several users are experiencing with the kernel version 3.18.x.  ^[link]
Just producing traffic on the SD-card produces something like this:

[ 1116.317108] sd 1:0:0:0: [sda]  
[ 1116.317199] Result: hostbyte=0x07 driverbyte=0x00
[ 1116.317220] sd 1:0:0:0: [sda] CDB: 
[ 1116.317233] cdb[0]=0x28: 28 00 13 7b 13 08 00 00 f0 00
[ 1116.317297] blk_update_request: I/O error, dev sda, sector 326832904

Here's a command i run on the NANO to trigger it:
 

root@nano:~# badblocks -e 100 -v /dev/sdcard/sd1

You can then check dmesg for errors.
The SD-card seems to disconnect, and after a second it reconnects with a new device-name each time (sda,sdb,sdc,sdd,etc.)

Dmesg:
 

root@nano:~# dmesg
[191722.800000] usb 1-1.2: USB disconnect, device number 11
[191722.810000] sd 2:0:0:0: [sdc]
[191722.810000] Result: hostbyte=0x01 driverbyte=0x00
[191722.810000] sd 2:0:0:0: [sdc] CDB:
[191722.820000] cdb[0]=0x28: 28 00 00 1e b5 60 00 00 f0 00
[191722.820000] blk_update_request: I/O error, dev sdc, sector 2012512
[191722.830000] sd 2:0:0:0: [sdc]
[191722.830000] Result: hostbyte=0x01 driverbyte=0x00
[191722.840000] sd 2:0:0:0: [sdc] CDB:
[191722.840000] cdb[0]=0x28: 28 00 00 1e b6 50 00 00 10 00
[191722.850000] blk_update_request: I/O error, dev sdc, sector 2012752
[191725.890000] usb 1-1.2: new high-speed USB device number 12 using ehci-platform
[191726.030000] usb-storage 1-1.2:1.0: USB Mass Storage device detected
[191726.050000] scsi host3: usb-storage 1-1.2:1.0
[191727.050000] scsi 3:0:0:0: Direct-Access     Generic  STORAGE DEVICE   0933 PQ: 0 ANSI: 6
[191727.050000] sd 3:0:0:0: Attached scsi generic sg0 type 0
[191727.520000] sd 3:0:0:0: [sdd] 15187968 512-byte logical blocks: (7.77 GB/7.24 GiB)
[191727.530000] sd 3:0:0:0: [sdd] Write Protect is off
[191727.530000] sd 3:0:0:0: [sdd] Mode Sense: 21 00 00 00
[191727.540000] sd 3:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[191727.550000]  sdd: sdd1 sdd2
[191727.560000] sd 3:0:0:0: [sdd] Attached SCSI removable disk
[191727.890000] buffer_io_error: 31 callbacks suppressed
[191727.890000] Buffer I/O error on dev sdc1, logical block 557056, lost sync page write
[191727.900000] JBD2: Error -5 detected when updating journal superblock for sdc1-8.
[191727.910000] Aborting journal on device sdc1-8.
[191727.910000] Buffer I/O error on dev sdc1, logical block 557056, lost sync page write
[191727.920000] JBD2: Error -5 detected when updating journal superblock for sdc1-8.
[191727.950000] EXT4-fs (sdd1): couldn't mount as ext3 due to feature incompatibilities
[191727.960000] EXT4-fs (sdd1): couldn't mount as ext2 due to feature incompatibilities
[191728.020000] EXT4-fs (sdd1): recovery complete
[191728.020000] EXT4-fs (sdd1): mounted filesystem with ordered data mode. Opts: (null)
[191728.300000] Adding 1004364k swap on /dev/sdcard/sd2.  Priority:-4 extents:1 across:1004364k