Jump to content

There's always room for Pi.


cooper

Recommended Posts

I'll include those benchmarks in my post-construction tests for further evaluation.

My goal isn't to make a cluster that's great at computing (although that would be a nice result) but instead I want to investigate the scalability of certain pieces of software we make where I work and how this can be improved. If the network is slow (as they report having trouble with in the article you linked) any problems will simply arise sooner. I see no reason why the Odroid's computational prowess would dwindle when placed within a cluster, other than network or heat issues (like most modern hardware, it throttles itself when it gets too hot) which I think my setup can eliminate to the maximum extent possible.

Link to comment
Share on other sites

  • Replies 107
  • Created
  • Last Reply

Top Posters In This Topic

I took yet another look at the switches I would use and found that the cheapest one that fits the bill for me is in fact the Netgear GS308 (the gray ones in the image) which is slightly cheaper, slightly thinner and according to the linked page it only draws 3.64w max (although they put 12V 0.5A on the device. Still better than the 12V 1A of the 108).

I also found someone on an ebay lookalike website offering the 16-port Netgear GS116 so I bid the price of the 2 GS308s combined. If I win, this'll be sorted and the width drops to 28.5 cm which I would like, and I'd have 2 additional ports to play with. We'll see how that works out, in the mean time I'll assume the 2 GS308s will be the way to go.

As I mentioned in another topic I was investigating using an IKEA LEKMAN Mini Chest to house my little project. Unfortunately this one won't work because there's insufficient vertical space. The outside dimension of the chest is 330x170x250 (WxHxD in mm). Deducing the height of the Odroid U3 at 20mm exactly from a screengrab of an introductory video of the Odroid U3, the switches being 27mm and the PSU being 860mm if I were to cram everything on top of eachother (which, indeed, would obstruct the PSU's airflow) I'd get 153mm (remember, I'm planning on 2 layers of odroids). Then I need to add room for the inner framing and the outside frame of the chest. It might just fit, but still without airflow for the PSU. So I think I need a box with an inner size is at least 31cm wide (because of the switches), 20cm deep (the ITX board is 17 and I want room to mount a fan behind it) and 20cm high.

Let's see what can be found with that...

Edited by Cooper
Link to comment
Share on other sites

I'm starting to think that I should just get a wooden plank. cut that up and go DIY all the way.

Something like this beech wood panel for 12.50 euros cut into 2 pieces of 325x250x18 and 2 pieces of 210x250x18. That would make a box with outside dimensions of 325x246x250mm (WxHxD) and an all around thickness of 18mm which should be thick enough to put some screws into. Some chunks of metal for the corners and I'll be sorted.

And the ebay lookalike thing came through swimmingly. With a little luck I should have that GS116 in my posession tomorrow for a mere 65 euros. :-)

Edited by Cooper
Link to comment
Share on other sites

Thinking of power supplies again. Check out this puppy. From just one of those I can power 2 Odroids off of 12v at 93+ % efficiency (expected power draw of an Odroid in full swing with idle gpu is 1.5-1.6A)! It's 2cm high, but so is the odroid so I'l still happy.

Only drawback is I'm going to need a voltmeter because the output voltage is set by turning that little screw on the blue brick and there isn't exactly a 'setting' for 5v. Then again, I'm pretty sure I'll find a way to make good use of that aswell plus I found one for under 10 euros on deal extreme again (they're true to their name, I'll give 'em that).

Link to comment
Share on other sites

Thinking of power supplies again. Check out this puppy. From just one of those I can power 2 Odroids off of 12v at 93+ % efficiency (expected power draw of an Odroid in full swing with idle gpu is 1.5-1.6A)! It's 2cm high, but so is the odroid so I'l still happy.

Only drawback is I'm going to need a voltmeter because the output voltage is set by turning that little screw on the blue brick and there isn't exactly a 'setting' for 5v. Then again, I'm pretty sure I'll find a way to make good use of that aswell plus I found one for under 10 euros on deal extreme again (they're true to their name, I'll give 'em that).

You do mean 93% of a 83% power supply. getting you near 80% or something in total ;) About the voltage. You have a litle room, But your best getting as close as possible to that point :)

Btw where are you ordering those odroids ? Any dealers in the eu ?

Edited by GuardMoony
Link to comment
Share on other sites

The GS116 arrived last night so part 1 is in.

My_GS116_zpsd6e7d8fa.jpg

Regarding the power solution yet again, I just saw this buck converter which can provide 5V15A. Since I plan to put 6 boards (expected 1.6A, max 2A) on a single power rail I could use one of these to power the full power strip leading up to the board trays on a single shelf. I'd only need one and I can place it somewhere where I can easily guarantee sufficient cooling (hell, I could put a fan straight on if I feel like doing so).

My current layout would be something like this:

- Top layer, 32.5cm inner width, 3cm height, all used up by the netgear which gives it 2 cm on either side for ventilation (there are ventilation slits in the box there).

- Two center layers about 3cm high each containing a frame into which I can slide 2x3 trays containing 2 odroids each. At the back of the tray it will slide into a 5V power rail, make contact and from there feed the Odroids. Each tray will be U shaped and hold the Odroid in such a way that the inside width of the U will be at least 85mm. I don't yet know how thick I want the tray material (probably plexiglass) to be, but I think it's safe to assume the width of the tray itself will be something like 90mm. The frame it slides into will be a grid and the borders will be of the same material so that's another 5 mm to separate the 3 trays, total width 275mm which leaves 50mm on one side. My plan is to use this space to mount some hooks which will be used to route the network cables.

- Bottom layer, 10 cm high, housing the ATX PSU and the ITX board side by side. That will account for 32 cm width leaving a little room for spacing. Because of the height I'll mount the mobo elevated and house the 2 harddisks below it which should keep the cable clutter to a minimum. Obviously the PSU would need to be modular so I had to rethink the previously chosen make and model and I'm now going for the CoolerMaster G550M. Its fan is always on (though not spinning loudly until truly loaded) so I could mount the buck converter to the ceiling of this layer, in the intake airflow of the PSU which might need some shrouding to guide that flow.

I originally thought I could house the cluster in a thinner box due because the switches defined the width and I had a more compact solution now, but I discovered that in the overall placement of things I had put the PSU in sideways which for a number of reasons won't work. Thus, placing the PSU and the 17x17cm ITX board next to one another it adds up to 32 cm so I included 5mm of spacing just to make sure everything's put in safely. I may have to mount some hooks on the bottom under the mobo aswell to be able to tie the mobo cable down as it's 56cm long and the expected distance needed is roughly half that...

Edited by Cooper
Link to comment
Share on other sites

The only place I know of that sells them is hardkernel themselves and they do so at or near cost. Also, because they always ship via FedEx the shipping fee is high AND you must pay import tax which adds up (it's like 20% or some such).

One 65 dollar droid is about 50 euros but shipping is about 20 euros and tax is another 10. Total price for one: 80 euro give or take. A little less when shipping in bulk.

EDIT: Found a review by a fellow dutchie who got a U2. He said import tax is 30-40% of the price of the package (which I think includes the shipping fee for some reason) so the single unit price tag is about 95 euros.

EDIT2: Went to the website and compared for the quantities I'm considering. The shipping fee is 50 for 6, 70 for 12. It amounts to 440 or 860 dollars respectively. Dollar to Euro is currently about 0.725 so I expect an actual rate of 0.75 though I'll probably get shafted a little more than that. At that rate, and assuming 40% import tax, the prices become 462 or 893 which results in a per-unit price of 77 or 74 euros. To put that in perspective, I can get a Raspberry Pi for a little under 40 euros so it's fair to say that one odroid delivered to my doorstep (well, actually my work address since they only deliver during the day) costs the same as two Raspberry Pi's picked up at a nearby computer shop. In terms of performance per currency I'm still winning, but not by as wide a margin anymore.

Edited by Cooper
Link to comment
Share on other sites

I only just managed to look at that URL you posted (It's blocked at work due to it being a 'malware' site... sigh). Looks fun. :)

Last night I made a partial little mock-up for a tray that would hold 2 Odroids in orange and which is assumed to be 3cm high total which is more than it needs to be since the Odroid itself is just 2 cm high so 2.5 ought to suffice, but I disgress.

You'll have to imagine the sides of the tray being there aswell. It would be open at the top, except for maybe at the back for structural support another strip across between the back Odroid and the black dots. I might cut out holes under the Odroids aswell to prevent heat buildup though that might not be necessary.

tray_mock_2_zpsb6876fe2.jpg

tray_mock_1_zpse8689f9b.jpg

When I initially imagined the tray, I was thinking of a nice handle at the front from which you would pull out the tray but I couldn't find a nice handle of appropriate width, moderate cost and I'm worried about it snapping off. So I decided that instead of a handle, I'm going to attach a small knob. I'm currently considering either this rather fancy one or one of these really cheap ones. Since the former is a full 5cm wide and probably protrudes as much from the tray aswell it's probably not going to be the one. Looks nice though. The second is a set of boring 22mm round knobs on a short stand.

My plan, as you can see from the mock-up, is to have a U-shape where the knob attaches to the tray. Looking at it though, I had it pegged at 1cm wide so it can contain the head of the screw that would keep the knob in place. My problem with this is that it obstructs a fairly large chunk of the front opening (11.7%) and this part is in front of where the heatsink resides. So I'm now thinking of, instead of using a top-to-bottom U shape, I'm using a front-to-back H shape where that center bar is in fact 2 bars between which the screw would go. This allows me to put the vertical bars of the H closer together as the head of the screw can simply rest against them rather than needing to be within that structure. It's probably also better at withstanding the forces at play.

To keep the Odroids in place I'm planning to take some large flathead plastic screws and glue them head-down on the tray. The Odroid would simply slide over them. I could put a bolt on top but currently don't see a need to. Gravity would almost certainly suffice for keeping them in place. At 18 euros for 50 screws I'm still looking for cheaper alternatives that would preferably not involve me drilling into the plexi and not increase the height of the tray by keeping the board off of the tray bottom more than what's required (there's a 4mm audio plug there keeping it off the floor anyways).

Something else I had imagined was two slanted bits of plexi that would support the sides without blocking the airflow much, and could be used as cable gutters to get the network cables out. I think I'm going to keep them around for the structural support, but I think I'm going to put two of them in as an inverted K on the right-hand side since that's where the network cables will come out. Might even do that on the other side aswell for structural support of that strip at the top and because symmetry just tends to look good to me.

tray_sketch_1_zpsae107657.jpg

Which brings me to those two black dots. This is the power solution. I'm planning to use rivets here, pretty side below the board and one of those lollipop shaped metal things into which the wires to the power connector of the boards would tie. The idea is that when you slide the tray into the frame, there will be rivets there aswell using a similar lollipop-shaped metal connector that ties into the 5V and ground wires respectively. The pretty side of the rivet is simply a thin, curved metal shape and my plan is that the tray will just fit over it and when fully inserted has its rivet center just a tad beyond the opposing rivet's center. It would keep the stress down, gravity would retain the connection and when you want to pull it out you'd need to put a bit of force in to get the tray over the opposing rivet before it comes out and until this happens the electrical connection is maintained.

Finally, I was thinking about how to deal with the network cables. It has to look good but most importantly it has to be functional. Previously I thought that, since I have about 5cm of excess space on one side I could make a nice cable gutter for all the wires there. Problem with this idea is that if I need to extract a tray, It needs to come forward by, going from the mock-up, some 15 cm. That means that the cables to that tray needs to be completely free and probably loose from the switch before you can pull it out. If it's the bottom-left tray, that's an awfully long bunch of cable I'd need to be able to separate. It just doesn't seem practical. So instead of that idea I'm going to have 3 sets of small vertical gutters right next to the trays. Each gutter would only hold a set of 4 cables (5 where the one from the Mini-ITX board at the bottom shelf comes up) and it would neatly tie into the switch. That way I can also make do with just 3 different colors of cables rather than 7. One color for the 2 cables from the top tray, one for the 2 from the bottom one and the third and final color for the ITX. It would be easy to separate the two sets if one tray needs to come out and when you take a set of 5 ports on the switch, going from the left, the first 2 are for the top tray, the next 2 are for the bottom tray and the remaining 1 would be empty (available to plug in for monitoring) leaving one port open to be used by the ITX. It would look good and visually separate things aswell which is even better.

Edited by Cooper
Link to comment
Share on other sites

The Odroids will find their root partition on the ITX, but for the actual booting you must have some form of storage to house the kernel as by itself the Odroid has no options in that regard (no EEPROM on the network adapter or built-in flash storage of any kind). The Odroid provides 2 options for this: MicroSD and eMMC. The latter is very, very fast and thus, obviously very, very expensive. So I just placed an order with AliExpress for a lot of 30 128MB MicroSD cards.

For 47.50 dollars (approx 35 euro) they will send me:

30x 128MB MicroSD card

30x MicroSD to SD adapter card (a.k.a. garbage)

30x plastic case for adapter and card.

2 or 3 cardreaders

I wasn't expecting the cardreaders and I have an external one already plus one in my laptop, but no harm to have a few spares.

The reason I went with 30 is simply because I expect a few duds (it's another pile of plastic crap from China which gets sent via regular mail... my expectations are kinda low even though the seller is reputable) and the sd card will effectively decide the 'identity' of the Odroid board. Based on its MAC address, which is set during boot so it's in the boot script which resides on the SD card, the ITX will present it with a different root filesystem. So if I want to turn an Odroid into a database server I simply have to insert the sd card with a mac address associated with a database node. Want a webserver instead? Replace the sd card with one for a webserver. Having a LOT of these cards gives me a big set of options to fiddle with the tasks distributed throughout the cluster.

It'll take a month or so to arrive, but that makes part 2 sorted.

I'm hoping my company bonus will be high (against better judgement) as that would speed up this process somewhat, but it's still not looking like I'll be acquiring the Odroids before august. I might get the ITX before that time, though. I know I have the sheets of plexi I intend to use in storage at home so once the girlie has taken a hike this should pop up and allow me to work on that aswell.

Edited by Cooper
Link to comment
Share on other sites

In response to questions asked on the Odroid forum I was pointed at a german reseller who sells the Odroid-U3 from Germany at 80 euro a pop. If you're doing only 1 or 2, this is probably cheaper to order from when you're in Europe. For my volume, going the direct route will probably save me a bit.

The super cool thing about that reseller though... He also sells the wire and plug to power the board and for a fairly reasonable price to boot. Nice!

Link to comment
Share on other sites

Drew a mock-up, at scale, of the front-view that I'd envisaged. I'm a little worried about the bottom-right. I thought I could place 2 harddisks side by side below the Mobo, but each HD is just over 10cm wide. It would mean the two HDs would stick out by 3mm. The solution I drew in the mock-up has the two harddisks on top of eachother, leaving about 4cm for the Mobo which might just be enough, but things are mighty cramped there, airflow will be poor... Unless I can get a bit of room in between the parts, this might simply be insufficient.

If there were the case, I'm going to tack on a few strips of wood at the back to widen things to the point where the 2 harddisks can lay there side by side. I figured I'd have to do that anyways, but just to have a path into which the back panel with the 2 14cm fans would slide. Now all of that might have to move back by half a centimeter too...

Clustermock-upfront_zpsaf58504c.png

Edited by Cooper
Link to comment
Share on other sites

Although it's over twice the price of the original chunk of wood I can get a Scots Pine panel of 1450x250x18mm which should allow to do everything I want from it. One of the 4 edges (length-wise) is smoothed off for some reason, so on that side I'll cut the width down to probably something like 23cm, allowing 2 harddisks to be positioned side by side with a little bit of spacing. Maybe I can even get the guys to cut that 2cm strip off in such a way that I end up with a long, 0.5 cm wide strip that will allow me to easily create the groove the back panel needs to slide into.

Something else I looked into today. Those cable from the PSU to the MoBo and associates will end up WAAAAAY to long. I'm contemplating chopping the excess off and attaching a new ATX connector, possibly repeating the process with the SATA connectors aswell. It does add another 25 dollars or so to the total bill of the device, though, and looking at the current BOM it's something I need to keep an eye on...

Cluster_Bill_Of_Materials_zps2dca47bf.pn

Edited by Cooper
Link to comment
Share on other sites

Swift update. BOM was missing the MicroSD cards, costing 35 euros, so the project total currently stands at 1165 euros.

I've recently ordered the ring terminal connectors, the 5v15A buck converter and the multimeter making the current total amount already spent just under 133 euros.

Something else that occurred to me. I will switch the PSU and the ITX+HDs around. The PSU has its MoBo cable coming out of it on the left so routing it to the MoBo will become that much cleaner when on that side the actual board is aswell. Also, I need to feed the power solution from the PSU and the best place to run the required wires up without showing too much of it is using that chunk of unused space on the right of the mock-up.

Link to comment
Share on other sites

  • 2 weeks later...

Oh, I do intend to make them. Plenty of them. But I'm currently awaiting delivery of some chunks of chinese plastic and such. Once that's in I need to wait for the girlie to take a hike (estimated at half july) at which point I will clean up my house (2 weeks) and then really get cracking. I should have found my sheets of plexi then too so at that point I will order my Odroids which will take about a week to arrive (this stuff is FedEX only, so it's expensive but swift) and then really get cracking.

Link to comment
Share on other sites

As an aside, I took a look at the possible use of SFX form factor PSUs. The idea is viable, but the only modular SFX PSU is an expensive one. 100 euro plus. I don't feel it's worth it.

The other alternative is this TFX PSU which should fit and at under 60 euro for a modular 80 plus gold psu... That's interesting.

Link to comment
Share on other sites

So I got a nice little present in the mail yesterday:

sd_cards_box_zps660ef145.jpg

The sticker says it's a 20 dollar gift which is somewhat accurate except for the fact that it's a commercial purchase of something with a price tag of 47.50 dollars (35 euros) but anything over 20 euros requires an import tax so by doing it this way it simply becomes even more of a gift to me. :)

Contents:

sd_cards_and_reader_zps52d6369b.jpg

I was surprised at the USB card readers they included. I expected some clunky large usb stick that you would stick the card in at a 90 degree angle or something. Shows how little I know. These things are AWESOME! The MicroSD stick is inserted in this usb plug, disappearing almost entirely inside it and you (obviously) stick the reader in your computer where IT disappears almost entirely.

I love compact and this is about as compact as it gets. And I've been given 2 of them! :)

The rest is your standard 30x128MB MicroSD cards with SD adapter sheet. First reaction of my coworkers: "Why did you buy only 128 MB? That's, like, 2 photographs or something."

Answer: This was 1.60 dollars a piece. 2GB was about 3 dollars. I'm not going to pay an additional 42 bucks for storage I really have no use of.

The way this is going to work is that I'm going to have to probably put colered stickers on the sticks themselves to keep them apart and mark the plastic box that contains the card with what that particular card is for. Might use a sticker for that aswell. I'll give the private cluster network its own Class B network segment where the /24 subnet will define the type of node (and, thus, the color of the sticker) and the last 8 bits are for node id within that subnet which I'll just write on that sticker. So if I use green for my database nodes and this sd card is for node 14 that card will contain a green sticker with the number "14" written on it.

I might have to invest in a magnification glass...

Oh, and about that TFX format PSU - I read a review about it and the more I learn about this thing the more I love it. I would probably even go for the non-modular one as it's cheaper and the 'modularity' with this type basically boils down to 1 plug for the ATX connector and 1 plug for everything else. So I'd either be butchering the 'everything else' plug, or I'd try to hide the cables a bit. I think I'm going for the latter. The 5v side is rated at 80W max with sufficient amps available for 6 Odroids in full swing but it's a little close. I'll have to keep an eye on it at least initially. Might have to go for a second buck converter on the 12v line.

I'm going to redo my at-scale mockup to see what impact this is going to have. I know already it will result in the cluster becoming thinner and less high, plus I intend to use the screws for the PSU fan as mount point for a bracket that will hold the buck converter in place directly in front of the air intake so it can benefit fully from the airflow coming through here. Interesting bit though, in the review the fan didn't kick in until the system was under 150 watt load. What that means is that if I yank out any odroids, I should try to mainly pull out the ones attached to the buck converter as its temperature isn't monitored and if that overheats....

Edited by Cooper
Link to comment
Share on other sites

And here's the revised mock-up front view of the cluster:

Clustermock-up2front_zpsabcd70d5.png

It's gone from an outside dimension of 361x226mm to 326x212mm and instead of having the two hard disks on top of each other with the MoBo above that the horizontal space that's become available on the horizontal plane allows me to mount the 2 harddisks with the short side to the front (so cables at the back) side by side, leaving ample room for the MoBo above it and because in this image the PSU sucks its air in from the right there's some additional airflow there which, as I mentioned, I intend to put the buck converter into between the mobo and the psu.

To mount the harddisks to the outer wood I will cut some strips of plexi that will run across the long sides of the bottom of the harddisks. The strips will attach to the harddisks using the bottom mounts so I need to drill some crevices in the wood to allow the screw head some room, and the strips will go beyond the end of the disks so that when it's on the wood a chunk sticks out that I can run another screw through to attach it to the wood. This will also produce the space I want to have between the harddisk and the bottom.

Works for me.... :)

Edited by Cooper
Link to comment
Share on other sites

And of course I need to throw another spanner in the wheel. These Odroids can get a tad warm. Normally you can just let them run in an open space without any active cooling, but in a compressed setup like this some active cooling is recommended. In case of overheating the core automatically throttles itself, but I don't pay good money to let my cpu cores tell me it's too warm. So I want to run some low noise fans at the rear to get some good airflow going.

I can put in 2 140mm fans as per the BOM but this also requires a lot of vertical space. So much in fact that the fan would cover the PSU exhaust. That's not that much of a problem except for the fact that I think it's just looks terrible and it would degrade the design. Next I looked at putting in 3 80mm fans covering the Odroids, which would work but would product airflow ONLY on the Odroids. The rest of the chassis would have to fend for itself. And you have to understand that I intend to mount the fans on an otherwise closed plexi backplate so any heat needs to get out either using the fan's airflow or on its own accord via the front. So next I drew in 3 92mm fans and it looks excellent. I completely cover the odroids themselves and, if I rotate the PSU 90 degrees, doesn't bother the PSU. One drawback with the rotated PSU is that in the horizontal plane there is now exactly 3 mm of combined free space between the outer walls, the PSU and the 2 harddisks. I'll use that to space out the harddisks so the PSU goes dead against the wood and the harddisks get 1mm spacing left and right. These aren't 'enterprise' harddisks or anything. An added bonus to tilting the PSU is that I can further reduce the height of the setup and retaining some room for the buck converter (also moving with the PSU so it's now above it though still in front of the air intake) in a perfect position to receive some airflow from the rear fans ensuring proper cooling regardless of the activity of the PSU fan.

But as with everything, there's a price to pay for changing your mind and in this case it's that 3 92mm fans cost more than 2 120mm fans. I had budgetted this for 15.60 but 3 Arctic F9 92mm comes to 19 euro.

Which reminds me. About that PSU: I've decided to go for the BeQuiet TFX Power 2 300 instead which, like the Seasonic, is also TFX (identical dimensions), also 80Plus Gold but a little more potent on the 5v line while being marginally cheaper to boot. It's still more expensive than the originally budgeted 46.10 euro ATX PSU but at 55,70 for a Gold-rated PSU (the other was Bronze) that's, I feel, defendable.

Link to comment
Share on other sites

Well thus far all changes have been caused by finding out the original idea simply wouldn't work so it's not _that_ bad.

The one mistake at least I'm not making is to say "I want X" and rush out to buy the various parts for it only to find out later that half the parts don't fit as expected and the parts chosen actually can't do what you expect them to do.

Link to comment
Share on other sites

So did you look into getting the power cables?

How can I power on the board?
It doesn't support the USB power in any case. Please use the power supply to the DC-Jack.
We recommend 5V 2A adaptor. Inner diameter is 0.8mm, outer diameter is 2.5mm. Center is Positiver and Outer is Negative. Normally ODROID-U3 consumes about 1A - 1.5A in most cases.

What is the minimum peripherals to run ODROID-U3?
- HDMI monitor for output device & Mico-HDMI Cable (Not included)
- Mouse for input device (Not included)
- MicroSD Card for storage of OS & user data (Not included / 8GB or higher is required)
- MicroSD Card reader to transfer OS image and data (Not included)
- Power supply: DC 5V (Not included)
Something like:
2.99$ each but 16$ shipment
or
there 4.99$ but no shipment. ( reported as gift ) and get tracing for 3$
but its pretty expansive for a cable :(
Edited by GuardMoony
Link to comment
Share on other sites

I bugged the Odroid forum for info on the exact plug type. I went with these. 2,5 euro for 20 (I only need 12) and I'll just source the cables locally. I'm thinking of getting something like speaker cable since at the DIY store at least I could only find black wire. A spool of thread that's 10 meters long costs a little under 5 euro so this should work out to a total under 10 euro, if need be by simply using the one cable I need to get for the power solution anyways.

I've ordered that batch of plugs on the 22nd so it should arrive any day now.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recently Browsing   0 members

    • No registered users viewing this page.

×
×
  • Create New...