The *BEST* microSD card for a Pineapple

[cooper]

Since I only recently got my Pineapple I haven't done all that much with it, but I noticed in the videos about the device that I've seen so far people are really going out of their way to emphasise that you should pair it with a good Class 10 MicroSD card. Which makes me wonder what cards people have bought for it and why.

Also, what size would people recommend? I'm thinking 16 ought to do it, but that's just a wild guess, really.

Looking at the concept of a 16GB Class 10 microSD card, I can find no less than 57 different brands...

I'm currently leaning towards the SanDisk Extreme Pro, not because it's the most expensive one but because with the 1.4.0 release video Seb mentioned swapping. If the device is doing that, you're going to get hit by the achilles heel of microSD cards: random writing IO. And it just so happens that according to a pretty recent test by Tom's Hardware the SanDisk Extreme Pro is ahead of the pack by a very healthy margin indeed. The other thing to note in that benchmark is that a larger card performs rather a lot better than the smaller one. Look at the Kingston UHS-I U3 32gb and 16gb cards. Identical except for their size. Oh, and the 50% drop in write performance.

So, what card(s) did you get and how happy are you with it(them)?

[Trolljegeren]

I just put a 32GB Sandisk Ultra class 10 in mine a couple of days ago, though I haven't really done anything with it yet other than install 1.4.0 and some infusions.

I have the same card in my Galaxy S2, though I didn't do any research or testing when I chose these. It was mainly down to what was cheap and had good reviews on Amazon!

[Merlintime]

I picked up a SanDisk Ultra 32GB MicroSDHC Class 10 UHS Memory Card for use with the Pineapple. So far it was working great.

[DrDinosaur]

I got a SanDisk class 10 card 8GB for about $10.

[cooper]

I don't think I'm going to discover much with this topic. It would seem people specifically went out to by a class 10 for their pineapple and after that never looked back. Result is that there's no reference for people to judge their microSD card by.

So here's what I'll do:

I'm going to take the fio tool written by Linux Block layer maintainer Jens Axboe and I will use that to measure the performance of any microSD card I can get my hands on.

It just so happens that my mom is very much into digital video and she's got a stack of all sorts of MicroSD cards. Few were bought with actual performance in mind - when she needed a card she bought one that she thought would fit the bill and I wouldn't be at all amazed if she picked them for reasons like their color (although I know a couple at least truly FLY). What this does mean is that it should give me a very diverse testing field.

I'll holler back the results and we'll take things from there.

The fio tool uses an input file that describes what it should do. I'll use the provided input file that's meant for testing SSDs. You can read here what the various options mean.

[global]
bs=4k
ioengine=libaio
iodepth=4
size=1g
direct=1
runtime=60
directory=/mountpoint/of/sd/card
filename=ssd.test.file

[seq-read]
rw=read
stonewall

[rand-read]
rw=randread
stonewall

[seq-write]
rw=write
stonewall

[rand-write]
rw=randwrite
stonewall

Here's the rather lengthy result of running this against the (crap) harddisk in my laptop so you get some idea of the numbers this tool will produce:

~ $ fio ssd_test.fio 
seq-read: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=4
rand-read: (g=1): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=4
seq-write: (g=2): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=4
rand-write: (g=3): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=4
fio-2.1.9
Starting 4 processes
seq-read: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [___w] [59.1% done] [0KB/1176KB/0KB /s] [0/294/0 iops] [eta 02m:24s]  
seq-read: (groupid=0, jobs=1): err= 0: pid=9211: Fri May 23 10:03:29 2014
  read : io=1024.0MB, bw=29559KB/s, iops=7389, runt= 35474msec
    slat (usec): min=0, max=1053, avg=19.79, stdev= 4.96
    clat (usec): min=0, max=252096, avg=530.48, stdev=1390.13
     lat (usec): min=0, max=252102, avg=551.09, stdev=1390.29
    clat percentiles (usec):
     |  1.00th=[    0],  5.00th=[  306], 10.00th=[  326], 20.00th=[  346],
     | 30.00th=[  370], 40.00th=[  454], 50.00th=[  470], 60.00th=[  482],
     | 70.00th=[  494], 80.00th=[  516], 90.00th=[  764], 95.00th=[  828],
     | 99.00th=[ 1448], 99.50th=[ 4512], 99.90th=[10560], 99.95th=[16512],
     | 99.99th=[32128]
    bw (KB  /s): min=18216, max=120632, per=100.00%, avg=29619.87, stdev=11781.42
    lat (usec) : 2=2.61%, 100=0.01%, 250=0.13%, 500=69.80%, 750=17.22%
    lat (usec) : 1000=7.75%
    lat (msec) : 2=1.78%, 4=0.19%, 10=0.41%, 20=0.09%, 50=0.02%
    lat (msec) : 100=0.01%, 500=0.01%
  cpu          : usr=8.72%, sys=22.59%, ctx=259701, majf=0, minf=27
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=262144/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=4
rand-read: (groupid=1, jobs=1): err= 0: pid=9214: Fri May 23 10:03:29 2014
  read : io=50000KB, bw=852949B/s, iops=208, runt= 60027msec
    slat (usec): min=13, max=85, avg=24.99, stdev= 3.53
    clat (usec): min=205, max=213627, avg=19172.78, stdev=19534.25
     lat (usec): min=229, max=213656, avg=19198.62, stdev=19534.22
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    4], 10.00th=[    4], 20.00th=[    6],
     | 30.00th=[    8], 40.00th=[    9], 50.00th=[   12], 60.00th=[   16],
     | 70.00th=[   23], 80.00th=[   31], 90.00th=[   45], 95.00th=[   60],
     | 99.00th=[   93], 99.50th=[  108], 99.90th=[  135], 99.95th=[  153],
     | 99.99th=[  176]
    bw (KB  /s): min=  708, max=  948, per=100.00%, avg=833.33, stdev=48.44
    lat (usec) : 250=0.04%, 500=0.03%
    lat (msec) : 2=0.17%, 4=11.34%, 10=35.01%, 20=21.31%, 50=23.98%
    lat (msec) : 100=7.36%, 250=0.75%
  cpu          : usr=0.39%, sys=0.69%, ctx=12502, majf=0, minf=25
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=12500/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=4
seq-write: (groupid=2, jobs=1): err= 0: pid=9216: Fri May 23 10:03:29 2014
  write: io=1024.0MB, bw=22419KB/s, iops=5604, runt= 46771msec
    slat (usec): min=0, max=9685, avg=21.54, stdev=34.49
    clat (usec): min=0, max=516125, avg=700.93, stdev=4979.81
     lat (usec): min=0, max=516133, avg=723.31, stdev=4989.67
    clat percentiles (usec):
     |  1.00th=[    0],  5.00th=[  290], 10.00th=[  326], 20.00th=[  370],
     | 30.00th=[  498], 40.00th=[  524], 50.00th=[  540], 60.00th=[  548],
     | 70.00th=[  572], 80.00th=[  588], 90.00th=[  756], 95.00th=[ 1400],
     | 99.00th=[ 4256], 99.50th=[ 7840], 99.90th=[16512], 99.95th=[23680],
     | 99.99th=[391168]
    bw (KB  /s): min= 8369, max=86632, per=100.00%, avg=23002.38, stdev=7912.58
    lat (usec) : 2=2.26%, 50=0.01%, 100=0.01%, 250=0.09%, 500=28.15%
    lat (usec) : 750=59.34%, 1000=3.81%
    lat (msec) : 2=4.78%, 4=0.56%, 10=0.85%, 20=0.09%, 50=0.05%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%
  cpu          : usr=7.42%, sys=17.21%, ctx=254836, majf=0, minf=23
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=262144/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=4
rand-write: (groupid=3, jobs=1): err= 0: pid=9218: Fri May 23 10:03:29 2014
  write: io=68236KB, bw=1137.1KB/s, iops=284, runt= 60009msec
    slat (usec): min=10, max=1259.1K, avg=161.76, stdev=11423.51
    clat (usec): min=453, max=1267.9K, avg=13899.95, stdev=30048.59
     lat (usec): min=523, max=1267.1K, avg=14062.57, stdev=32189.76
    clat percentiles (usec):
     |  1.00th=[  604],  5.00th=[  676], 10.00th=[ 8384], 20.00th=[ 9920],
     | 30.00th=[11072], 40.00th=[11584], 50.00th=[11968], 60.00th=[12480],
     | 70.00th=[13632], 80.00th=[15808], 90.00th=[19584], 95.00th=[21888],
     | 99.00th=[30080], 99.50th=[36096], 99.90th=[602112], 99.95th=[749568],
     | 99.99th=[1269760]
    bw (KB  /s): min=   40, max= 2344, per=100.00%, avg=1183.20, stdev=361.23
    lat (usec) : 500=0.01%, 750=5.72%, 1000=0.34%
    lat (msec) : 2=0.38%, 4=0.25%, 10=13.35%, 20=71.84%, 50=7.78%
    lat (msec) : 100=0.10%, 250=0.02%, 500=0.04%, 750=0.10%, 1000=0.04%
    lat (msec) : 2000=0.02%
  cpu          : usr=0.55%, sys=0.93%, ctx=17030, majf=0, minf=21
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=17059/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
   READ: io=1024.0MB, aggrb=29559KB/s, minb=29559KB/s, maxb=29559KB/s, mint=35474msec, maxt=35474msec

Run status group 1 (all jobs):
   READ: io=50000KB, aggrb=832KB/s, minb=832KB/s, maxb=832KB/s, mint=60027msec, maxt=60027msec

Run status group 2 (all jobs):
  WRITE: io=1024.0MB, aggrb=22419KB/s, minb=22419KB/s, maxb=22419KB/s, mint=46771msec, maxt=46771msec

Run status group 3 (all jobs):
  WRITE: io=68236KB, aggrb=1137KB/s, minb=1137KB/s, maxb=1137KB/s, mint=60009msec, maxt=60009msec

Disk stats (read/write):
  sda: ios=274923/279349, merge=65/424, ticks=565400/1385160, in_queue=2652568, util=99.69%

I don't know precisely what all the numbers mean but we can work on that when we have some actual numbers to compare with.

I intend to run the tests using the USB MicroSD cardreader I got with the MicroSD cards I bought for my Odroid cluster project. I'll repeat the first test I perform with the SD cardreader built into my laptop and one of those adapter things (it's just plastic and as such this adapter won't affect the results). If there's a measurable difference I'll make a note of that and complete the test with the fastest of the two.

Edited May 23, 2014 by Cooper

[Oli]

All seems a bit futile to me. SD cards are pretty much commodity devices.

Get a class 10 SD with a capacity sufficient for what you require from a reputable brand. End of story.

Typically I'd bet the SD is not the bottleneck so even if you find the "best" SD card the performance difference over a $10 class 10 SD from Amazon wouldn't be all that great.

[cooper]

If it doesn't matter as you suggest, why even bother with a Class 10 then?

Class 10 means nothing more than a minimum of 80 Mbit/s (or 10 MB/s) transfer speed guaranteed in sequential operation. In practical terms it refers to sequential write since sequential read can go WAY faster compared to writing due to the nature of the storage medium. There are cards out there that can do way better than this. What that means in practical terms I guess we'll just figure out but like I mentioned in the post that started this topic, the Pineapple needs to swap due to the fact that it only has 64MB of RAM. If you want to run nginx, python, php and ruby all at once I'm guessing that'll be a tight fit and so the swap comes into play. If you can make the accessing of that swap faster the performance of your Pineapple will increase.

Since the nature of swapping in my experience is anything but sequential that whole Class X stuff goes straight out the window. Here's an example. Look at the top of page 5. In his test the Class 2 card just tears past the Class 10 when it comes to writing data randomly rather than sequentially. Do you really think you're not going to notice such differences?

Edited May 23, 2014 by Cooper

[Oli]

The class of SD card isn't always a true reflection of real world speed, but it is an indicator -> class 10 is typically going to be a lot faster than a class 2.

It's the law of diminishing returns. You could have the fastest SD card in the world - but unless you have other hardware to take advantage of it - so what, you are bottlenecked by other components of the system.

The AR9331 is only a 400MHZ SOC with pretty limited RAM and probably not the faster protocol for reading/writing to the card. Will spending the rest of your life trying to get an extra percentage point increase in SD card I/O really help much? In my opinion, no. Get a class 10 SD card from a reputable brand and that is good enough.

If disk I/O was the bottleneck, the story would be different: for example, old hard drives were the bottleneck in PCs -> hence an SSD drive is a good investment.

[cooper]

You could have the fastest SD card in the world - but unless you have other hardware to take advantage of it - so what, you are bottlenecked by other components of the system.

Which is kinda the point I'm trying to make. There are currently no less than 65 32GB microSD cards I could buy with prices ranging from 5.5 to 65 euro. 26 of those are Class 10's with prices ranging from 12 to 30 euro. Since I think we both agree that the reported maximum sequential operation speed doesn't matter all that much, who's to say that that 5.5 euro card, a Class 4 Transcend, doesn't outperform the 12 euro Kingston Class 10?

class 10 is typically going to be a lot faster than a class 2.

Typically, perhaps. But if you would please just look at that link I posted previously, page 5, you would see that of the 8 cards he tested the Class 10's have a random read performance only 25% above that of their tested Class 2's whereas their random write performance was about 1% that of the Class 2s tested. To me it's very much NOT a done deal, and I'm not suggesting people run out and buy the most expensive card out there. I'm suggesting people run out and get the card that provides the actual performance befitting their usage pattern. And you can't know which one that is without a) knowing your usage pattern and b) how a reasonable number of cards react to specific usage patterns.

My plan is to work on b.

Edited May 23, 2014 by Cooper

[Sebkinne]

On the hakshop, we offer Sandisk Ultra 16 and 32GB cards with the WiFi Pineapple Builder.

These cards have proven to be reliable and fast for the WiFI Pineapple MKV.

Best Regards,

Sebkinne

[cooper]

I'm still going to measure them.

[Oli]

Well good luck with that Cooper - you obviously have a lot of time on your hands!

Make sure you run tests actully from the pineapple though so that you have the SD I/O bandwidth and the AR9331 processor / RAM to make it a fair test.

If you find a significant deviation from the "buy a class 10 from a decent brand" advice I will be shocked.

For anyone else, just buy the ones Seb recommends and support Hak5 or go to Amazon, search "sd class 10" and buy the best rated one with the capacity that is required (should you already have a Pineapple and need a better card than the stock one).

[cooper]

you obviously have a lot of time on your hands!

No comment

[xrad]

Well good luck with that Cooper - you obviously have a lot of time on your hands!

I don't think Cooper sleeps.

[Darren Kitchen]

/etc/init.d/cooper sleep

[cooper]

I'll write things up in a new topic when I have some more stats, but here's a quick listing of the 5 cards I've tested so far. Note that these tests were performed on my laptop rather than the Pineapple - I'll get to that later.

+-------------------------------+--------+--------+------+------+
| Card                          |  SeqR  |  SeqW  | RndR | RndW |
+-------------------------------+--------+--------+------+------+
| Pineapple original 2GB        |  3852  |   848  | 3002 |    6 |
| SanDisk Ultra 4GB             |  5797  |  2346  | 4004 |  206 |
| SanDisk Ultra 8GB             |  5760  |  1650  | 3853 |   13 |
| SanDisk Extreme 32GB          |  6064  |  2846  | 3595 | 1142 |
| SanDisk Extreme 64GB*         | 12214  | 13459  | 1897 | 3060 |
| ICIDU Class 4 16GB            |  7169  |  1422  | 4914 |  260 |
| ICIDU Class 10 16GB           |  6470  |  2415  | 5281 | 1377 |
| Samsung Evo 16GB              |  9021  |  3327  | 6682 | 1381 |
| SanDisk Ultra 16GB            | 13232  |  2675  | 3779 |  925 |
| Transcend Class 10 16 GB      | 13503  |  2752  | 4941 | 1456 |
| Transcend Class 10 32 GB FAT  |  9254  |  7602  | 4120 | 1028 |
| Transcend Class 10 32 GB Ext4 |  9154  |  7576  | 3791 |  851 |
| Kingston Class 10 16 GB       | 12778  |  1042  | 4268 |  904 |
+-------------------------------+--------+--------+------+------+

* This card was exFat formatted so accessed via fuse.

Things to note:
* Presented values are in kilobytes per second.
* The block size is 4kb which is pretty close to worst case possible when it comes to writing. Real-life performance is likely to be a bit better than this, but not by that much.
* The stock Pineapple card I got behaved wonky during testing and died afterwards, so mistrust the numbers there.
* I am aware that this table means the tool shows that some cards are performing below spec for sequential access. I don't have an explanation for that.

* The two Transcend 32 GB cards were kindly tested by Byter on his machine running Kali linux using fio 2.0.8 where all other values were found by me using fio 2.1.9 and the Ext4 formatted card had already seen some use whereas the FAT card was brand new. The filesystem used has *NO EFFECT AT ALL* on the numbers produced by the tool.

Edited June 3, 2014 by Cooper

[cooper]

I added the last bunch. I'll have 3 more 16-giggers for testing on tuesday, specifically a SanDisk Ultra 16GB, a Kingston class 10 and a Transcend class 10.

What I find interesting in the test so far is that in general getting a solid brand doesn't get you all that much additional performance. We'll see when the other cards are available if that makes that much difference. The two non-SanDisk ones combined cost almost as much as the single SanDisk.

The other thing not proven by this test but somewhat suggested is that in these devices size really does matter. I've seen tests on review sites that show that going from 16 to 32 gig gets you about 80% more performance while the price goes up by about 50% which sounds like a mighty tempting deal. Moving on to 64 gig tends to not add all that much anymore. The results I got from the SanDisk Extreme 64GB tends to refute that claim, but I don't trust the involvement of FUSE in this...

One last thing. A microSD card is very much not like an SSD. Wear-leveling, if done at all, is rather more basic and in general a cell on one of these cards will last for about 1000 writes. If you're going to use it for swap or some other use that involves continued (re)writing to the device, expect that you'll have to replace that card every so often since they do wear out.

Edited May 24, 2014 by Cooper

[cooper]

I had a go at building fio for the pineapple today. It wasn't much of a success. First mistake I made was to assume the cpu was a little-endian mips32r2. On executing the fio binary I'd made there was some crap sent to the screen and my ssh locked solid - had to kill it from another session. I copied one of the existing binaries onto the sd card and compared them from my laptop where I noticed my mistake. About half an hour later I had a new toolchain and tried again. Everything _seemed_ to look good, but then this program wasn't supposed to produce any screen output the way I ran it. All it was supposed to provide was a report file and that only at the end of its run, which took 6 to 8 minutes on my laptop. After about 30 minutes I started to keep track. After another 90 I went in with another ssh session and verified using top that the program was eating the cpu whole. Guess that explains why it took way longer. About 2 hours later (we were celebrating my dad's birthday so I tried to not constantly ssh home to see what's up) without change I looked at the mounted sd card directory and found no testing file present. The program had managed to end up in an endless loop before creating it.

I tried to run the program without the --output parameter redirecting screen updates to a file and sure enough nothing showed. Guess the cross compilation woes are upon me. I'll need to look into it and see what's up.

One thing I did note during compilation of fio is that it has some networkjng code in it (no idea why) which supports ipv6 and failed to find the constant for any address. I dug a little into the problem but quickly figured the fastest route to success was to remove the engine/net.c from the Makefile which indeed worked wonders during compilation.

I'll set things up in accordance with the guide best I can and will see if I want to do things differently. For starters, I really do want a full Linux on there rather than busybox in all its splendor. I'm thinking I'd also prefer the boot rom as something that just holds a kernel and a bootloader and everything else comes from the sd card. It has its pros and cons and can totally understand why it currently works as it does, but I'd like to try the alternate route as I'm more familiar with that. We'll see how that pans out.

[cooper]

Added the last three.

Main thing to notice here is how well the Transcend performs, considering it's one of the cheapest in the bunch. I was disappointed by the Kingston which cost the same as the Transcend. SanDisk I expected to perform better but in the random writing test it falls well behind the pack.

I'm still considering getting a 32 gig ICIDU to test if bigger is faster, and I want to do something to test this on from the Pineapple itself as that's where the numbers really count. All I can test for certain is the seq-read and seq-write using DD but I would very much prefer using fio on there aswell...

[Darren Kitchen]

Awesome findings, thanks Cooper!

[Oli]

Yep - testing on the pineapple is what i'm interested in. This is where the AR9331 speed and SD implementation come into play and where the bottleneck likely is.

[Byter]

Added the last three.

Main thing to notice here is how well the Transcend performs, considering it's one of the cheapest in the bunch. I was disappointed by the Kingston which cost the same as the Transcend. SanDisk I expected to perform better but in the random writing test it falls well behind the pack.

I'm still considering getting a 32 gig ICIDU to test if bigger is faster, and I want to do something to test this on from the Pineapple itself as that's where the numbers really count. All I can test for certain is the seq-read and seq-write using DD but I would very much prefer using fio on there aswell...

Today I bought some Transcend 32 GB Class 10 HC I cards for both PineApples. Since my focus is not storage benchmarking I don't have much knowledge on that subject. But always willing to perform some tests if you have intructions.

Formatting and copying infusions, including dependancies, went smooth.

PM me if I can be of help.

[cooper]

Thanks to Byter we now also have the 32GB Transcend's results in the table. It would appear that particularly with sequential write the larger card benefits. Random not so much, but the numbers are still pretty good.

I'm still looking to test at least one 64GB card because this larger size makes it an SDXC as opposed to an SDHC card and I want to know if that makes any difference.

Something else I'm currently considering is taking multiple readings for each card. The current table is produced by running each test (SeqR, SeqW, RandR, RandW) for 1 minute and then computing the numbers. I want to know if there's (a lot of) variance between repeat-runs since some numbers are particularly high whereas others are suspiciously low. I'm also going to consider setting a ramp_time on some tests to see if the deviation can be brought down a bit. As an example, the Sandisk Ultra's random write score was 925 but its min and max values are respectively 21 and 1560. That's clearly a HUGE difference, so maybe we can remedy that a bit.