CIA Drones used buggy, stolen code.

[SirNitram]

Link

The CIA is implicated in a court case in which it's claimed it used an illegal, inaccurate software "hack" to direct secret assassination drones in central Asia.

The target of the court action is Netezza, the data warehousing firm that IBM bid $1.7bn for on Monday. The case raises serious questions about the conduct of Netezza executives, and the conduct of CIA's clandestine war against senior jihadis in Afganistan and Pakistan.

The dispute surrounds a location analysis software package - "Geospatial" - developed by a small company called Intelligent Integration Systems (IISi), which like Netezza is based in Massachusetts. IISi alleges that Netezza misled the CIA by saying that it could deliver the software on its new hardware, to a tight deadline.

When the software firm then refused to rush the job, it's claimed, Netezza illegally and hastily reverse-engineered IISi's code to deliver a version that produced locations inaccurate by up to 13 metres. Despite knowing about the miscalculations, the CIA accepted the software, court submissions indicate.

IISi is now seeking an injunction to ban Netezza and the CIA from using the software or any derivative of it, in any context.

The relationship between the two firms dates back to 2006, when IISi signed up to resell Netezza data warehousing kit combined with Geospatial.

The code allows users, for example, "to incorporate and cross-reference vast amounts of business data with geographic location within the same database, and enable events (such as... a cell phone signal moving from one tower to another) to be matched with personal characteristics in the database (such as... the identity of the person whose cell phone signal has moved from one tower to another)", according to IISi's court filings.

Such techniques - quickly combining intelligence with live mobile phone surveillance from the air - are reportedly central to the CIA's targeting of missile strikes by unmanned aircraft.

They want to kill people with my software that doesn't work

The partnership between the two firms strengthened, and in August 2008 Netezza acquired exclusive rights to distribute Geospatial, alongside its NPS hardware. By August last year, Netezza was starting to promote its next generation appliance, TwinFin. Whereas NPS was based on IBM's Power PC chip architecture, the TwinFin relies on cheaper x86 silicon. As a result, Geospatial would not run on the new gear.

Nevertheless, Netezza sales staff sold Geospatial running on TwinFin to a "US government customer", which later turned out to be the CIA. The purchase order, totalling $1.18m, via an obscure Virginia IT consultancy, came through on 11 September last year. This despite - as claimed in IISi court documents - that the software product referred to on the order "in fact did not exist".

Up to this point IISi had done little work porting Geospatial, as its engineers had not had physical access to a TwinFin. Indeed, the agreement between the two firms did not require IISi to support the new machines - a fact confirmed last month by a Boston judge - but it agreed to begin the process in September 2009.

Netezza supplied the software firm with TwinFin hardware on 1 October. Within a week, Richard Zimmerman, IISi's CTO reported that porting Geospatial was "proving fraught with difficulties" and would take at least two months.

Two days later, on 9 October, the relationship took a strange turn. Jon Shepherd, Netezza's "general manager, location-based solutions" called Zimmerman to pressure him to deliver the code quicker, court documents say.

"He basically told me the CIA... wanted to use [Geospatial] to target Predator drones in Afganistan and that, quote/unquote, it was our patriotic duty to work with them to get [Geospatial] ported to the TwinFin as fast as possible and that we need to have a phone conversation the next day to discuss that," Zimmerman said in a sworn deposition to the court.

During a conference call the next day, Netezza CEO Jim Baum repeated Shepherd's claims that national security demanded IISi's help, according to the deposition. Shepherd suggested the CIA would accept untested code in chunks, Zimmerman said.

"My reaction was one of stun, amazement that they want to kill people with my software that doesn't work," he said.

According to the affidavit of IISi CEO Paul Davis, who was also on the conference call, his firm did not previously know Netezza had sold the undeveloped product, let alone for deadly application by the CIA.

In an email to Baum two days later, on Columbus day 2009, Davis wrote: "Jon [Shepherd's] statement, apparently endorsed by Jim [Baum] that the customer can 'just work with whatever we give them' is not consistent with how IISi works. And we don't really believe that is how our national security agencies work. Frankly, that response suggests a cavalier sales approach to a profound issue. Lives are at stake."

Enter Skip

Yet according to Baum's response, that is how the CIA worked. "It is the CUSTOMER who has indicated that he is willing to work with IISi and Netezza to accept code progressively," he wrote.

As a follow up, Davis got a call from a man who identified himself as Skip McCormick, of the CIA, to discuss speeding up the port of Geospatial. Davis was recuperating from a heart attack and could not speak at any length. Straight after the the call, however, he received an email from McCormick with a CIA address.

"We depend on the Geospatial tools here every day," it said.

"We just upgraded to a [TwinFin], but it doesn't yet have the Geospatial tools. I'm trying to figure out what options are available for getting them asap."

Davis had doubts the contact was genuine but The Register has established that a Hays W. "Skip" McCormick III, co-author of a 1998 book on software project management, has worked at the CIA for several years. Sources including conference guest lists record his involvement in software projects at the agency. According to book publicity he previously worked as a consultant to DARPA, Northrop Grumman and the Office of the Secretary of Defense.

Further evidence of the CIA's apparent acceptance of untested software is offered by an internal Netezza email from the same day as the crucial conference call. "A US Gov customer is expecting the toolkit to be available as soon as Monday for use in a mission-critical project," wrote project manager Razi Raziuddin.

"They do understand we won't have a fully-qualified, production-ready release and are OK with it."

Immediately after IISi's refusal to deliver untested Geospatial code, internal emails disclosed to the court show Netezza executives began making alternative arrangements. "I want to set up some time on Thursday to get on the phone with you guys to talk about some options in the event we need an alternative TwinFin solution," Shepherd told Netezza engineers in an email.
Thank God for optimists

On the Thursday one of the engineers told Jim Baum via email that "it appears" Geospatial was working on TwinFin. On Friday it emerged that however Netezza adapted the software, the results were inaccurate.

"For some strange reason many of the calculations are a little off, from 1 to 13 metres," wrote Joe Wiltshire, a federal account manager at Netezza.

"The customer is not confident they can live with the uncertainty in meters unless we can tell them a bit about why this is happening."

"No matter how you slice this, we are likely screwed," Netezza CEO Baum replied.

The unreliable results were traced to a floating point problem, but less than a week later Wiltshire reported to Shepherd that in fact "they are satisfied" and believed "the minor discrepancy in metrics... is due to [TwinFin] doing a better job".

"Thank God for optimists," came Shepherd's reply.

The solution was later referred to as "the spatial toolkit hack" in Netezza emails when it began producing further errors in November. The existence of the hack, and its use at the CIA was only revealed after Netezza sued IISi, claiming it breached its 2008 contract by refusing to port Geospatial to TwinFin.

That case was dismissed last month, with the judge finding that contrary to Netezza's repeated claims, IISi was under no obligation to carry out the work. Discovery also revealed that Shepherd had called on staff to develop "our own version of the spatial toolkit", which was introduced in January this year as "Netezza Spatial", which is available on the open market.

Now IISi claims both the hack and Netezza's own software are illegally based on reverse engineering and misappropriation of its trade secrets, and is pursuing an injunction that if granted would block their use by anyone. It's unclear which, if either, is currently in use at the CIA. A hearing on the injunction application is scheduled next week.

The complex case, which has so far received scant press attention, has the potential to embarrass the CIA, and the White House. President Obama has significantly expanded use of clandestine drone assassinations, despite heavy criticism from the UN and others.

Questions remain over whether repeated Netezza claims that the CIA needed Geospatial for drone assassination operations were correct, and the full truth is unlikely to be made public. However, the suggestion the agency accepted a rushed job and saw inaccuracies in an optimistic light is likely to draw further controversy to the programme.

Netezza and IISi both declined to comment for this story. A CIA spokeswoman said the agency does not comment on pending litigation, especially if it is not a party to the lawsuit.

You heard that right: Missing by up to 13 meters was considered part of a better programming job. I increasingly hate the idea of armed drones after this...

[Sea Skimmer]

13 meters is still less then the CEP of a JDAM; and any other guided weapon will be aimed with a laser or optics so someone has to see and mark the target. Its also less then the lethal radius of almost all our air dropped weapons. If you think this is some specific reason to hate UAVs you just don’t know what you are even thinking about. For a human pilot who can clearly see the target and using CCIP dive bombing the CEP of the bomb salvo will easily be several hundred meters. Having a guy fly the plane isn't an advantage for accuracy, its an advantage for deciding to drop the weapon in the first place. Not the same thing.

In fact military planning for precision and unguided weapons has built in error budgets for target locating error, particularly with artillery and they can get pretty large. Before GPS 13 meter accuracy was nearly impossible for aircraft and cruise missile navigation. That wasn't that long ago that not everything had GPS. In the war against Serbia in 1999 Predator drones flew in combat without any GPS capability for example. It was added just in time for September 11th to make it vital.

That’s why high precision requires a laser guided bomb or missile, with a human who sees and aims the laser. This is done exactly the same on a remotely piloted vehicle as it is on a jet with a pilot and WSO. In fact that is one of the vast advantages of a UAV, you can put more then one person onto the UAV video feed and have a dedicated second guy for aiming weapons, while most jet fighters are single seat without the weapons system officer in the back.

[Zaune]

Further to Sea Skimmer's point, so far as I can make out from the article, this software is not being used to guide the weapon itself; it's providing an approximate location for the target. I can't imagine that direction-finding on someone's cellphone is accurate to within the CEP of a Hellfire II anyway, so I very much doubt the UCAV operators are ever cleared hot without additional surveillence or human intelligence.

[Skgoa]

You are both missing the point IMHO. They used stolen and buggy software in their hightech weapons. The problem is not, wether or not it was still good enough. ITS THAT THEY DID THAT.

[Hawkwings]

Disregarding the stolen part, the buggy part is excusable. It works. It's good enough. Good enough means good enough. The CIA would rather have a good enough product now, than some polished miracle product some indefinite time later.

[Sea Skimmer]

You are both missing the point IMHO.

No you miss the point. Nitram didn't say a word about that in his one liner and he's shown a track record of having zero conception of modern military operations in the past. A excellent example being when he tried to denounce the ABL program on the basis that the 474 laser plane had to burn fuel to fly.

[Highlord Laan]

"You stole our software!"
CIA: "Here's a cool million. Now shut the fuck up."

For some reason the idea of a civilian company trying to drag the CIA into a courtroom over a civil matter strikes me as...unwise. Not that I'm in favor of such things, but intentionally annoying one of the more dangerous government agencies in the country is not something I'd put on my to-do list.

[Steel]

Disregarding the stolen part, the buggy part is excusable. It works. It's good enough. Good enough means good enough. The CIA would rather have a good enough product now, than some polished miracle product some indefinite time later.

That is not true. In a computer program, especially something that would be in a low level language like this product, if there is a bug (and the fact it sometimes gave the wrong answer guarantees there is a bug) then ANYTHING can happen. Yes sometimes it may manifest as being out by only 10m, but that same bug might put you out by 10km other times.

The danger is even greater if people have just hacked around in the program they had without access and understanding of the source code. Christ knows what might happen then. It could be that if you enter a targets name longer than 12 letters in their version it targets somewhere totally random. You can't know, because you haven't debugged and tested it fully.

[Aaron]

"You stole our software!"
CIA: "Here's a cool million. Now shut the fuck up."

For some reason the idea of a civilian company trying to drag the CIA into a courtroom over a civil matter strikes me as...unwise. Not that I'm in favor of such things, but intentionally annoying one of the more dangerous government agencies in the country is not something I'd put on my to-do list.

Nothing so dramatic, the government will just pull something out its ass to block the suit.

[Phantasee]

CIA agents are for abroad, lawyers are for domestic issues. Pretty sure the CIA can't do stuff on US soil?

[SirNitram]

You are both missing the point IMHO.

No you miss the point. Nitram didn't say a word about that in his one liner and he's shown a track record of having zero conception of modern military operations in the past. A excellent example being when he tried to denounce the ABL program on the basis that the 474 laser plane had to burn fuel to fly.

Whatever you'd like to tell yourself, I guess. I mentioned the being off by up to 13 meters being, as the article says, considered part of better programming. Here, let me quote it to show you how full of it you are:

"For some strange reason many of the calculations are a little off, from 1 to 13 metres," wrote Joe Wiltshire, a federal account manager at Netezza.

"The customer is not confident they can live with the uncertainty in meters unless we can tell them a bit about why this is happening."

"No matter how you slice this, we are likely screwed," Netezza CEO Baum replied.

The unreliable results were traced to a floating point problem, but less than a week later Wiltshire reported to Shepherd that in fact "they are satisfied" and believed "the minor discrepancy in metrics... is due to [TwinFin] doing a better job".

It's not that they are off by 13 meters and that's unforgivable, it's that they're happily accepting stolen code.. And think the bugs that MAKE it less accurate is because it's 'better'! But keep harping on what you think I mean.

[MKSheppard]

I mentioned the being off by up to 13 meters

So fucking what?

A 500 lb guided bomb (like a JDAM) has a 100% lethal radius of 20 meters; and a 50% lethal radius of 60 meters.

The percentages are how many people die on average instantly when caught in the lethal radius.

We've already begun to deploy thermobaric warheads on the hellfires that our drones and helicopters fire -- in one of the first combat uses of them, the guy who dropped it compared it's blast and explosion to a 2,000 lb bomb going off.

So thirteen meters is an acceptable programming error.

[SirNitram]

I mentioned the being off by up to 13 meters

So fucking what?

A 500 lb guided bomb (like a JDAM) has a 100% lethal radius of 20 meters; and a 50% lethal radius of 60 meters.

The percentages are how many people die on average instantly when caught in the lethal radius.

We've already begun to deploy thermobaric warheads on the hellfires that our drones and helicopters fire -- in one of the first combat uses of them, the guy who dropped it compared it's blast and explosion to a 2,000 lb bomb going off.

So thirteen meters is an acceptable programming error.

Yea, it is. But, because you have the same premature ejaculation issue Skimmer has, you didn't get past that phrase to where I said they said it must be due to superior product. Take a cold shower, Shep.

[MKSheppard]

Yea, it is.

So what's the fucking issue here? We're using the product since it works well enough to put warheads on foreheads and blow up jihadis. So I fail to see a problem here.

[SirNitram]

Yea, it is.

So what's the fucking issue here? We're using the product since it works well enough to put warheads on foreheads and blow up jihadis. So I fail to see a problem here.

If you could read a full post without prematurely jerking off to your victory, you'd see it. Since you can't, not worth bothering further. Your blatant snipping in the last case is a little too in your face to be taken seriously.

[Lonestar]

Nitram believes in letting perfection being the enemy of good enough.

[SirNitram]

Nitram believes in letting perfection being the enemy of good enough.

Where did I say that, hmm? As oppose to viewing thinking an inaccuracy means it's a better product?

[PhilosopherOfSorts]

If its a smaller inaccuracy than with whatever they were using before, then I'd say it was a superior product.

[SirNitram]

If its a smaller inaccuracy than with whatever they were using before, then I'd say it was a superior product.

This is true. However, the only evidence is that they chalked up the error to a superior product, which is not at all sensible. With the fact these agencies are supposed to be able to handle modern, high-tech intelligence threats, this sort of stupidity leads one to doubt.

[Lonestar]

This is true. However, the only evidence is that they chalked up the error to a superior product, which is not at all sensible. With the fact these agencies are supposed to be able to handle modern, high-tech intelligence threats, this sort of stupidity leads one to doubt.

I fucking love it. The article states that the product's purpose is to rapidly cross reference databases and the CIA accepted the up-to 13 meter discrepency because it was doing a better job than the previous product.

Now, this may be my dumbshit gainfully employed defense contractor mind at work here, but it sounds to me that the new product is accessing and crossreferencing data much faster than the old one. If the real effect of the bug does not materially affect the other tools that would be used(as Skimmer and Shep helpfully pointed out, it doesn't) and it's otherwise compressing the time needed for the decision assistance making product to do it's work, than the CIA is not letting perfection being the enemy of good enough.

Especially if "good enough" *IS* better than the last iteration of the product, from the end user standpoint. Your argument really is that the CIA should allow perfection to be the enemy of good enough because, uh...there's a problem that does not materially affect the program!

[Spoonist]

For some reason the idea of a civilian company trying to drag the CIA into a courtroom over a civil matter strikes me as...unwise.

On the contrary they are forced to. Note that the article says that they where sued first by their business partner, then they counter sued. The evidence for the countersuit was encovered during the first suit trial.
I'm guessing that this is their only option from bankrupcy.

---

On the other issue, I'm sorry but I've got to agree with Nitram on this. Regardless of how useful the software kit turned out to be for the CIA it is a display of incompetency if the chain of events happened as in the article.

Whereas NPS was based on IBM's Power PC chip architecture, the TwinFin relies on cheaper x86 silicon. As a result, Geospatial would not run on the new gear.

So switching to a hardware the software was not designed for because it is cheaper.
Then selling it to the governement. No fault on the CIA yet but on the selling company selling a non-compatible unit.

"We just upgraded to a [TwinFin], but it doesn't yet have the Geospatial tools. I'm trying to figure out what options are available for getting them asap."

So the CIA bought the new kit, but note the 'upgrade' right there. If we take the article at face value, then it means that they where already using the old more expensive and working version.
So instead of reverting to the last point before failure they want to go ahead ASAP. Fault 1.

according to the deposition. Shepherd suggested the CIA would accept untested code in chunks, Zimmerman said.

"It is the CUSTOMER who has indicated that he is willing to work with IISi and Netezza to accept code progressively,"

Going ahead with untested software. Fault 2.

"For some strange reason many of the calculations are a little off, from 1 to 13 metres," wrote Joe Wiltshire, a federal account manager at Netezza.

"The customer is not confident they can live with the uncertainty in meters unless we can tell them a bit about why this is happening."

"No matter how you slice this, we are likely screwed," Netezza CEO Baum replied.

Here they do it right. They test it and find a flaw which they point out to the manufacturer. Here is where they should revert to the last point before failure. Since its different hardware they could have them running in paralell. One live and one in a test environment.
After all its only two months until a tested replacement would be in place.

The unreliable results were traced to a floating point problem, but less than a week later Wiltshire reported to Shepherd that in fact "they are satisfied" and believed "the minor discrepancy in metrics... is due to [TwinFin] doing a better job".

"Thank God for optimists," came Shepherd's reply.

WTF
They go from high end expensive hardware and move to cheaper hardware and the idiot think the change is due to the new unit working better!! Note that this is after talking to the head of development of the geostuff they rely on.
That is some fucking stupid rationalization right there as the "optimists" comment shows.
Fault 3.
Note that this also shows that the old unit is available, since that is what the discrepancy is compared to.

The solution was later referred to as "the spatial toolkit hack" in Netezza emails when it began producing further errors in November.

Which is exactly what any software developer worth his salt would expect of such a scheme.

[Lonestar]

WTF
They go from high end expensive hardware and move to cheaper hardware and the idiot think the change is due to the new unit working better!! Note that this is after talking to the head of development of the geostuff they rely on.

I reiterate what I said earlier, it's entirely possible that the actual speed for processing of data(for the end user) has been compressed, at least enough that a bug that does not materially affect the bombs of choice is pushed from the customer's mind. I hadn't caught the comment about the new hardware being cheaper. If the entire product is now cheaper, and that's the only difference(other than the meaningless "up to" 13 meters discrepency) then it was the right switch to make.

[Sarevok]

Time at hand was very limited and the rushed work on porting the software did produce results within acceptab parameters. So whats the problem here ?

[Vendetta]

Time at hand was very limited and the rushed work on porting the software did produce results within acceptab parameters. So whats the problem here ?

The rushed work on illegally reverse engineering the software and distributing a version in breach of copyright and most lilkely the contract between IISi and Netezza you mean.

Seriously, if this software played a little tune the RIAA would be suing the CIA for three times the GDP of the USA by now.

[Spoonist]

I reiterate what I said earlier...

So? What did that have to do with my post? Let me reiterate: "Regardless of how useful the software kit turned out to be for the CIA it is a display of incompetency if the chain of events happened as in the article."
So unless you can provide data deviating from the article its still incompetence regardless of results. Unless where you'd like to show where those steps I claimed faulty was a sign of competence.

If the entire product is now cheaper, and that's the only difference(other than the meaningless "up to" 13 meters discrepency) then it was the right switch to make.

You have no clue on how things like this works do you?
If it was untested there was no way to know if that is the only difference. There could be other errors elsewhere which proper testing would vet out.
Give anyone with experience in such development and they will probably say the same thing with more cursewords.

Time at hand was very limited and the rushed work on porting the software did produce results within acceptab parameters. So whats the problem here ?

If that was directed to me then you should note that the deviation was not within accepted parameters until the idiot thought that the deviation was due to the new unit being better (optimism snide) and falsely assuming the old unit was the one with the fault, when in fact it should be assumed after talking to the head of dev that it most likely was vice versa.