StarDestroyer.Net BBS

Just sent this rough design to the structural engineer;







To support gate operations it will have full electromagnetic shielding (double layered grounded copper screen, steel doors) and 25kW dedicated electrical power (considering a 48V double-conversion UPS setup). Will definitely need a full-size JCB for this one.

So is this for fun, or to be serious, because if its serious I wouldn't use copper mesh. The only place I have ever noticed that being used for real electromagentic protection of hardened facilities is as a door seal, rolled up on itself as a sponge like material. Welded sheet steel is a much more reliable form of protection for the cost for walls, ceiling and floor, and its easier to integrate with the required cable and pipe duct protection because said shield pieces can be welded into the walls directly, as can the interior shields for the surge protectors. Also got to make sure all the rebar is grounded or it turns into an amplifier. Welding it really helps but that's sure inflate cost. Flawless detail is far more important then any specific material. Suffice to say if your cables peforations are not protected the whole idea is bunk against a nuclear weapon, and any other threat can simply be dealt with by shielding specific pieces of equipment for a lot less trouble.

Sea Skimmer wrote:So is this for fun, or to be serious, because if its serious I wouldn't use copper mesh.

The copper mesh is for suppression of radio communications and EMI, not electromagnetic pulse, so I am not concerned about high voltage/high power transients (fortunately). Still have to be careful with cable & plumbing penetrations. For a planned experiment series (not the recreational stuff I talk about here) my preferred grid configuration is ~256 mini servers on a reliable isolated mesh network. Since we need a new garage anyway...

Then why don't you just put the servers in EMI shielded server racks, since you're going to need server racks anyway? That was what I was getting out. That should actually work better, because then you suppress some of the electronic noise generated within the room itself, which is probably going to cause as much data error as anything external ever would through your 'stock' double reinforced ~200mm class garage ceiling ever would. I'd still want the metal door of course, but it sounds like basic security would mandate one anyway. Then you don't have to worry about say, the sump pump emissions causing problems when it powers up, or heck emissions coming up the drain pipe or out of the ground, or how straight you make all those power cable runs after the power filter to avoid creating partial magnetic coils. Also then you don't have to worry about what happens when you merely open the door to the door.

Asidefrom that, how much of a heat load are you expecting? Conduction through basement walls can handle a lot of heat but 256 computers worth owning isn't exactly a mild amount of heat. Should be easy to do the calculations yourself, I've got references around if you want them for all of this, but you know, something easy to add during the design phase but very annoying to retrofit. I'd expect an engineer to address this themselves, but you'd still need to decide where the vent pipe comes out, and where the intake is since it shouldn't be in the garage interior. Monoxide ect...

Also since part of my present job involves fixing problems created by the ignorance of professional civil engineers, I trust nobody.

Sea Skimmer wrote:Then why don't you just put the servers in EMI shielded server racks, since you're going to need server racks anyway?

I'm a bit dubious on the utility of shielded racks in a compute application. I know you can get them for harsh industrial environments (lots of high-power motors putting out EMI), but that assumes you can fit all the sensitive equipment in one rack. If you have hundreds of 10G ethernet runs forming a mesh network, that is the biggest antennae and it is necessarily outside the racks. Cat 7 is double screened, but even still there are significant alien crosstalk concerns even with no external factors (in large cable bundles). The servers themselves have grounded casings complying with the usual EMI standards, although certainly some improvements could be made in the cooling openings etc. Still it is a sensible suggestion and I will see how expensive this is.

Aside from that, how much of a heat load are you expecting? Conduction through basement walls can handle a lot of heat but 256 computers worth owning isn't exactly a mild amount of heat.

Initially, 25 kW as limited by the single phase electrical supply. That is a 256 mini servers, 64 medium servers, or 16 beefy servers. The most effective hardware setup for the power and acquisition budget is determined by (a) the relative efficiency of GPGPU vs CPU load (for the compute tasks of interest), (b) performance cost of parallelism at several different logical levels and (c) actual cost and spec of the hardware options, all of which change quite regularly.

I'd expect an engineer to address this themselves, but you'd still need to decide where the vent pipe comes out, and where the intake is since it shouldn't be in the garage interior. Monoxide ect...

Airflow is not addressed on that initial plan. I was thinking 4 x inlets to floor level and 4 x outlets from ceiling level implemented via L-shaped 450mm precast concrete pipes from the basement level to ~50cm above exterior ground level (with weather caps), and grounded copper mesh screens at both ends. I would guess that forced ventilation is not necessary at 25 kW, but if it is the fans could be at the above-ground end of the pipe rather than in the server room; it would certainly be sensible to run power to the pipe caps for this eventuality. The large 'parts storage' area is actually the experiment area of course. The smaller room is for actual storage of inactive equipment (of which I have accumulated a distressing amount).

Starglider wrote: I'm a bit dubious on the utility of shielded racks in a compute application. I know you can get them for harsh industrial environments (lots of high-power motors putting out EMI), but that assumes you can fit all the sensitive equipment in one rack. If you have hundreds of 10G ethernet runs forming a mesh network, that is the biggest antennae and it is necessarily outside the racks.

You could still wrap a much smaller amount of mesh around the rack to rack cable runs, and not have the problem of being unable to attach anything to your walls because they are draped in this material. Also I'm just kind of suspecting that any contractor will try to charge you an insane markup to do this kind of installation, or else will have no idea what they are doing. If you use racks + possible cable shield all you need is a good ground, and you could take everything with you if you ever moved.

Cat 7 is double screened, but even still there are significant alien crosstalk concerns even with no external factors (in large cable bundles).

Ah but that was a point before of sorts, shielding the walls won't protect you from that. Some subdivided metal cable duct would help though. The amount of radiation that is going to get through a double rebar ceiling isn't going to be that high at that point.

The servers themselves have grounded casings complying with the usual EMI standards, although certainly some improvements could be made in the cooling openings etc. Still it is a sensible suggestion and I will see how expensive this is.

Can't hurt. Its a not irrelevant market, as most military and government stuff has gone to the rack approach as they no longer fear nukes as much as non nukes, and certainly a major commercial market exists too.

Initially, 25 kW as limited by the single phase electrical supply. That is a 256 mini servers, 64 medium servers, or 16 beefy servers. The most effective hardware setup for the power and acquisition budget is determined by (a) the relative efficiency of GPGPU vs CPU load (for the compute tasks of interest), (b) performance cost of parallelism at several different logical levels and (c) actual cost and spec of the hardware options, all of which change quite regularly.

I've never thought of single phase as a limit! I'm in the land of 120 safety volts and I work for people who have an extra electrical service just to run their hot water because their original service was maxed out by the central AC, and some other factors that are a tad retarded but can be summed up as 'utilities across other peoples properties suck'. Amazing house though, rare hawks in the backyard of a developed suburb are worth it. Couple USD a month extra for the second service charge.

I didn't actually look up the values but 25kw might really be pushing things, depending on just how hot you'll accept it running and worst case summers. I'm sadly drawing a blank on just which file it was has the figure I want to consult. Probably remember it while I'm at work inside the damn tower or something.

But look at it this way, I'm in a 13 foot wide two story + basement brick rowhouse in the US North East. My home heater, forced air twin gas burner, is rated at 80,000 BTU per hour. 25 KW is about 85,000 BTU an hour! That heater is very capable of keeping this place warm in in a 0 degree F winter, even with my 1950s level of insulation. And just by operating at all it keeps the basement nearly hot, and I have the basement vents all closed though of course the ductwork and the flue pipe run through it. You're talking about running such a thing constantly. not in bursts, in the summer. Some kind of powered vent seems very likely to be needed.

Airflow is not addressed on that initial plan. I was thinking 4 x inlets to floor level and 4 x outlets from ceiling level implemented via L-shaped 450mm precast concrete pipes from the basement level to ~50cm above exterior ground level (with weather caps), and grounded copper mesh screens at both ends. I would guess that forced ventilation is not necessary at 25 kW, but if it is the fans could be at the above-ground end of the pipe rather than in the server room; it would certainly be sensible to run power to the pipe caps for this eventuality.

I think running power to at least one is wise, no matter what the conduction math says, because heatwaves are a thing and humidity buildup could become a problem in its own right for short periods. It would also be useful for the resale value. I know building underground is a growing thing in the UK because EYESORES ect... so it wouldn't help to plan ahead for when you go senile and any children and or mutant alien sheep dependents need to rid your estate of this monstrosity. Also a powered vent would also give you the option of using a dust filter on the vent, which depending on how built up the area is, might be useful to keep all the computers cleaner.

Oh and make sure they give that planter a bottom slope and proper drainage! Several years of my life were partly spent fixing drainage problems caused by professionals with no common sense. Ever seen a french drain installed at the bottom of a downspout with underground run off pipe?

Sea Skimmer wrote:Ah but that was a point before of sorts, shielding the walls won't protect you from that. Some subdivided metal cable duct would help though. The amount of radiation that is going to get through a double rebar ceiling isn't going to be that high at that point.

Yes I will probably just get a lot of cheap copper plumbing tube, string it across the ceiling, ground it, and push the cables through it.

Initially, 25 kW as limited by the single phase electrical supply.

I've never thought of single phase as a limit!

The last time I checked (a few years back), the local power company will give you 120A @ 240V on a single phase, but if you want more they request that you use 2 or 3 phases, presumably for load balancing on the local substation. 415V comes as three phase 70 kW by default, but the install and equipment costs are a lot higher.

I'm in the land of 120 safety volts and I work for people who have an extra electrical service just to run their hot water because their original service was maxed out by the central AC

I am aware that residential electricity in the US is typically underpowered.

Some kind of powered vent seems very likely to be needed.

Well as you say I should certainly install it for hot days at least.

I know building underground is a growing thing in the UK because EYESORES ect...

Planning permission in the UK is generally Stalinist but non-inhabitable outbuildings don't have to have up-front approval if the roof height is under 3m or less. This is supposed to be for sheds and garages, but fortunatel there is nothing saying that you can't reproduce the evil supercomputer bunker from Superman 3 under said shed or garage. Of course you must still comply with a couple of kilograms worth (when printed) of building regulations and allow inspectors to come in and assess structural integrity.

Original Design Concept before budget review;

so it wouldn't help to plan ahead for when you go senile and any children and or mutant alien sheep dependents need to rid your estate of this monstrosity.

Based on current London building trends they would be more likely to extend it to a triple level basement. No joke, when I was looking at case studies on the reinforced concrete contractor web sites, several of them were 'extend single or double story central London iceberg basement into triple or quadruple level iceberg basement, allowing installation of bowling alley / shooting range / storage for a further 32 Rolls Royces etc'.

Oh and make sure they give that planter a bottom slope and proper drainage! Several years of my life were partly spent fixing drainage problems caused by professionals with no common sense. Ever seen a french drain installed at the bottom of a downspout with underground run off pipe?

Good point. The wife insisted on the planter.

If I were to ask just what exactly you were planning to do with an underground bunker containing a full server farm with heavy electromagnetic shielding, apparently as a hobby project, would I regret it?

Well after discussions with the structural engineer I think I am going to have to reduce the floorplan from 12m x 7m to 10m x 7m. The cost is reduced significantly with the rear wall further away from the house, due to avoiding the need to reinforce the wall as much during construction. The original staircase design was extravagant anyway, there will still be enough room for support equipment with a landing added on the upper level. Will be a little tight fitting 3 cars in the upper level, but not too bad for UK vehicles (smaller than US cars on average). He also thinks that interlocking precast structural blocks (with concrete pour fill) will suffice for the walls, cast RC is only needed for the floor / ceiling slabs.

Starglider wrote: Yes I will probably just get a lot of cheap copper plumbing tube, string it across the ceiling, ground it, and push the cables through it.

That seems like a better approach for the task.

The last time I checked (a few years back), the local power company will give you 120A @ 240V on a single phase, but if you want more they request that you use 2 or 3 phases, presumably for load balancing on the local substation. 415V comes as three phase 70 kW by default, but the install and equipment costs are a lot higher.

Yeah be much higher and make a lot of heat stepping back down. Around here you can get 240 V if you want it but they'd never put anything more onto a residential zoned property. It really doesn't make sense.

I am aware that residential electricity in the US is typically underpowered.

In the past it generally was because of use of 100 amp service boxes. Wattage wise that's still more power then you'd ever normally draw but it almost always meant certain curcuits in the house would be overloaded when something started. A lot of old services though could be upgraded to 200 amps, functionally the same wattage your talking about, because the wires were overbuilt. But sometimes that isn't the case and it can just be easier to wire in two services. it isn't common, but in the case in question the wiring run was rather long, probably approaching the limit possible for a single line with no pole. It was just the easiest and least disruptive way to do it.

Though we used to have one crazy guy near Philly with the Christmas lights from hell including an actual life sized santa claus monorail going in a loop whom paid every year to have four more 120V service boxes wired into his yard for a month each year and then taken away again, the power company probably had a real love hate thing going on with him. Just had the boxes on poles in the yard and wired the lights and displays directly into them. He got sued more then once over the traffic chaos this caused, but nothing was actually illegal about any of it...

Based on current London building trends they would be more likely to extend it to a triple level basement. No joke, when I was looking at case studies on the reinforced concrete contractor web sites, several of them were 'extend single or double story central London iceberg basement into triple or quadruple level iceberg basement, allowing installation of bowling alley / shooting range / storage for a further 32 Rolls Royces etc'.

I've read of five or six story ones, and being built with residential fire codes written for single story basements. Some day someone is going to drown from the spinklers over that.

He also thinks that interlocking precast structural blocks (with concrete pour fill) will suffice for the walls, cast RC is only needed for the floor / ceiling slabs.

That will be fine until an earthquake. Though it will also leak a lot worse if you have a surface water problem, but eventually solid walls would also leak, and the joints might always leak, the only real solution is finding somewhere else for the water to go ahead of time. So nothing wrong with blocks in any serious terms. Plus at least with all the heat from those computers you probably won't even need a dehumidifer for day to day moisture issues a block wall might have. It'd just all boil away. Still another advantage to a box + conduct approach vs the cage, a dehumidifer is another piece of equipment you could place in the room outside of the EMI shielding.

Zaune wrote:If I were to ask just what exactly you were planning to do with an underground bunker containing a full server farm with heavy electromagnetic shielding, apparently as a hobby project, would I regret it?

Given that he also works on AI, that doubly gives one a reason to worry.

Sea Skimmer wrote:Though we used to have one crazy guy near Philly with the Christmas lights from hell including an actual life sized santa claus monorail going in a loop whom paid every year to have four more 120V service boxes wired into his yard for a month each year and then taken away again, the power company probably had a real love hate thing going on with him. Just had the boxes on poles in the yard and wired the lights and displays directly into them. He got sued more then once over the traffic chaos this caused, but nothing was actually illegal about any of it...

I respect the man's dedication. One would think that with LEDs now you could do the same thing with a standard service.

I've read of five or six story ones, and being built with residential fire codes written for single story basements. Some day someone is going to drown from the spinklers over that.

I am sure Zaune will appreciate the irony. Although, it is more likely to be the cleaners than the owner as the later tends to be absent 80% of the time.

He also thinks that interlocking precast structural blocks (with concrete pour fill) will suffice for the walls, cast RC is only needed for the floor / ceiling slabs.

That will be fine until an earthquake.

London area maximum earthquake magnitude over the last century is only about 3.0 for most areas and 4.0 for the unluckiest parts of the south east. However;

Though it will also leak a lot worse if you have a surface water problem, but eventually solid walls would also leak, and the joints might always leak, the only real solution is finding somewhere else for the water to go ahead of time. So nothing wrong with blocks in any serious terms.

There is now an argument between a waterproofing contractor and a building contractor. The waterproofing contractor says that to comply with building regulations, concrete blocks (even aggregated interlocking waterproof) require both interior (tanking) and exterior (membrane) waterproofing, whereas waterproof RC requires only interior waterproofing, and that this more than eliminates the cost savings of using blocks for the walls.

I can probably save some money eliminating the integral cast concrete staircase and replacing it with blockwork supporting precast segments.

Starglider wrote:There is now an argument between a waterproofing contractor and a building contractor. The waterproofing contractor says that to comply with building regulations, concrete blocks (even aggregated interlocking waterproof) require both interior (tanking) and exterior (membrane) waterproofing, whereas waterproof RC requires only interior waterproofing, and that this more than eliminates the cost savings of using blocks for the walls.

I usually recommend lots of free-draining rock around the concrete leading to a tightline around the building that leads somewhere useful to deal with water, and coating the concrete with some kind of water coating or membrane. For a 2-story basement (!) it would be even more important to figure that out properly. That way you're (ideally) avoiding water pressure on the outside of the wall altogether.

Starglider wrote:[
I respect the man's dedication. One would think that with LEDs now you could do the same thing with a standard service.

Doubtful with the need to still run the house and the amount of mechanized stuff on the lawn. Probably get away with only one extra service though and a checklist style startup of the insanity.

London area maximum earthquake magnitude over the last century is only about 3.0 for most areas and 4.0 for the unluckiest parts of the south east. However;

We have nuclear plant built with such optimism! Realistically though if nobody is going to live in it who cares if it collapses.

There is now an argument between a waterproofing contractor and a building contractor. The waterproofing contractor says that to comply with building regulations, concrete blocks (even aggregated interlocking waterproof) require both interior (tanking) and exterior (membrane) waterproofing, whereas waterproof RC requires only interior waterproofing, and that this more than eliminates the cost savings of using blocks for the walls.

Not externally waterproofing a block wall is bullshit. And if the drainage is poor, by which I mean not exceptional, you will need that. Interior waterproofing does NOTHING if you don't have something on the outside. It seriously freaking can't. If the water already penetrated the wall it already penetrated the wall! Interior waterproofing is a way to keep down humidity from the wall being damp, not keep water from pooling on the floor from liquid leakage. It might be one thing if you were only going down part of one story, but a full basement needs proper waterproofing.

I kinda was wondering too how the block wall was going to be cheaper in this myself, as I always assumed it'd be waterproofed on the outside but poured concrete costs can depend heavily on the relative location of the ready mix plant.

I can probably save some money eliminating the integral cast concrete staircase and replacing it with blockwork supporting precast segments.

Totally. Probably be better quality too. A precast stair gets a chance to crack at the plant, one poured on site is down to the whim and skill of the person who built it. Hard to see any actual advantage to pouring a staircase on site if your only worried about a 4.0 earthquake. It might look sleeker I suppose.

For budget and scheduling reasons I have divided the build into three phases.

Phase 1 is the structural reinforced concrete shell, exterior waterproofing, drainage, passive ventilation and excavation / backfill. Completion of this phase is sufficient to use the structure as a garage and basic storage unit.







Drainage not shown on above plans. Phase 2 is exterior cosmetic treatment and interior stairwell and partition walls; establishes sensible access to lower area for further work and prevents wife from complaining about concrete shoebox in front garden.







Phase 3 will bring the facility to operational status, ready for installation of compute array.





The inclusion of underfloor heating is because I need to use the facility in the winter when the compute array is not loaded.

Zaune wrote:If I were to ask just what exactly you were planning to do with an underground bunker containing a full server farm with heavy electromagnetic shielding, apparently as a hobby project, would I regret it?

As I keep having to inform local law enforcement, 'Conspiracy to Commit Supervillainy' is not a crime in the United Kingdom of Great Britain and Northern Ireland.*

* It was decriminalised by Margret Thatcher in October 1986.

Hang on, you've got a room on there that is a combination kitchen/bathroom with no dividing door. I didn't think you were allowed to do that (every house I've ever been in has at least two doors between a toilet and the kitchen).

Starglider wrote:As I keep having to inform local law enforcement, 'Conspiracy to Commit Supervillainy' is not a crime in the United Kingdom of Great Britain and Northern Ireland.*

* It was decriminalised by Margret Thatcher in October 1986.

Well she would, wouldn't she?

Just remember; if the weakly-Godlike AI asks you if it has a soul, say yes.

Eternal_Freedom wrote:Hang on, you've got a room on there that is a combination kitchen/bathroom with no dividing door. I didn't think you were allowed to do that (every house I've ever been in has at least two doors between a toilet and the kitchen).

Maybe up north you have luxuries like separate rooms for your washing and your cooking, but this is London;



Actually it is very common for new build studio flats to have just one internal door, i.e. micro-bathroom and everything else in the same room. No doors at all is contrary to building regulations but some shady landlords do it anyway. Technically I would need a partition door there, although personally I'd prefer to just overpower the air extraction. Practically I will build the structure as uninhabitable and look at a conversion application later.

P.S. As part of an optional test protocol, we are pleased to present an amusing depressing fact; the floor area of this proposed garage / storage unit is 160% of the typical floor area of a new build three bedroom family house in this area.

I'd forgotten about studio flats (even though I lived in one) that get away with just one door. Though I'd hardly call what was provided a "kitchen."

And what do you mean, "up North"? I'm in the depths of the Welsh Valleys just north of Cardiff, it's not particularly fancy here (except for our internet connection, that's just wonderful).

What sort of air filtration are you looking at providing?

TimothyC wrote:What sort of air filtration are you looking at providing?

A layer of coarse then a layer of fine stainless steel mesh on the outside vent, two layers of fine copper mesh at the base of the pipe. That's for rodent / insect prevention and EM screening, I hadn't really considered particulate filtering, but I guess if I'm using forced ventilation a layer of 15mm car air intake filter foam would be fine. I will make the pipe caps unscrew easily for access to the fans and filters.

Starglider wrote:Maybe up north you have luxuries like separate rooms for your washing and your cooking, but this is London;...

Couldn't you put the toilet under the stair? Maybe you don't want to run plumbing all over, but that's close to the other room and it'd separate them.

Make sure to slope everything towards drainage (garage floor slopes to garage door, lower level slopes toward sump, roof slopes toward door unless you build a French drain somewhere else).

Any thought to drainage around your building yet? Soil is great at holding water and causing leaks.

That place is... how can anyone live there? That's a toilet next to a kitchen. It's a container for disposing the most vile and disgusting of human excretions and it's placed right next to where you prepare food. How have people not died?

Same way you don't die when you brush your teeth every morning, if you keep your toothbrush in your bathroom anyway. The vast majority of bacteria in human waste came from the humans who made that waste and will probably do very minimal harm. You might be surprised how much shit crawls around the kitchen anyway. Not to say that keeping things clean isn't important, and I do find that somewhat fishy, but as long as one washes their hands, that does a surprising lot.

Mind you, after having kids, poop loses some of its shock value...

Starglider wrote: A layer of coarse then a layer of fine stainless steel mesh on the outside vent, two layers of fine copper mesh at the base of the pipe. That's for rodent / insect prevention and EM screening, I hadn't really considered particulate filtering, but I guess if I'm using forced ventilation a layer of 15mm car air intake filter foam would be fine. I will make the pipe caps unscrew easily for access to the fans and filters.

Fiberglass heating/cooling system filters might be a better idea, and are after all actually meant for this sort of rile. Tend to result in much less restriction then the dense paper used for car filters, and while they aren't as effective against extremely fine dust that isn't really a problem for computers either. Realistically though a fine metal mesh screen might be all you really need and last forever. Think drier filter. Many things like that exist to choose from.