Special report According to the British government, the cloud will make everything better. The UK gov is rolling out something called G-Cloud, which according to No 10 means faster and more flexible IT procurement based on a list of open systems and approved standards. The punchline for the British taxpayer is that services will be delivered faster, at a reduced and a competitive rate.
To prove that the government gets "it" – "it" being the cloud – the minister whose department is driving G-Cloud, Francis Maude of the Cabinet Office, has even been to the west coast to press the flesh and meet those behind the interwebs.
More ReadingUK.gov to drive stake through heart of big IT outsourcing dealsUK.gov hires data-handling privatisation point man as CTOIs it all over for UK.gov's G-Cloud 3.0? A footnote in history awaitsIBM begs Britain's new top cops: C'mon, set up pre-crime unitsIt's official: No 10 mandates 'open systems' options for Sir Humphreys
Microsoft's Ferrar: too much cloud fever?
There, Maude met some of the tech companies most closely associated with cloud. We're told he eschewed the usual stops at government suppliers such as Oracle for firms including Cloudera, which delivers Hadoop services; MongoDB specialist 10Gen; cloud infrastructure specialist Joyent; and eBay's PayPal.
But G-Cloud doesn't necessarily mean what you might think it does. Number 10 hasn't floated an Amazon-style or Salesforce-style platform for developing and hosting applications or crunching Google-licious workloads. Nor has it rolled out an Apple-style AppStore or Android marketplace that downloads paid apps to the smartphones and fondleslabs of the nation's citizens, civil servants, doctors and teachers.
What G-Cloud consists of – for now, at least – is two things: a way to approve and certify software, hardware and services from technology companies – from handsets and calling plans to hosted email and collaboration services, and something the government calls Cloudstore, which lists the services and the contact details of their suppliers.
As catalogues go, though, this isn't exactly the last word and it seems as if the government's done a first-rate job of going where so many others have been before – avoiding the pitfall of buying into the branding rather than the practice. Sure, everything looks cloudy: everything is nicely boxed away as a software-as-a-service (SaaS); an infrastructure-as-a-service (IaaS); a platform-as-a-service (PaaS); or as "specialist cloud services" – and you also get things like variations of Microsoft's Office 365 or IBM cloud migration services.
But plenty of services come with huge gaps where suppliers haven't bothered to fill in details or where the government says "service is waiting to be assured". Still, at least Cloudstore does say "beta" on it – a moniker that world+plus dog slapped on its site after it was popularised by Google with Gmail, and one which has since become a lot less trendy. Oh, dear.
The British government is notorious for bringing in big swinging policies that its incumbents claim will change absolutely everything: health, housing and education have been meddled with and pulled this way and that as intellectual fashions and those espousing them have swept through Westminster. The changes don't always work out.
When it came to supplying big services and infrastructure to government, remember how the Private Finance Initiative (PFI) was going to change things for the better? It has now fallen out of favour, having supposedly cost more than expected.
Is cloud the latest manifestation of the change-of-direction disease that afflicts British government? The move to the "cloud" is, of course, being driven by that most fickle and historically shortsighted of movements, far more bound to over-promise and under-deliver than politics: technology.
Francis Maude is the "ministerial face" of G-Cloud, but one of the people developing and selling government's digital change agenda is a rising star in the party jointly running the coalition government. Formerly a practicing techie, Liam Maxwell has provided advice that has shaped the ICT policy of the Conservative Party and, therefore, the government. His role is to "provide expertise on how government can use innovative new technology to deliver better, cheaper solutions."
Inside the Cabinet Office with him is Chris Chant, a career Whitehall techie who's argued for change in how IT is delivered in government. As G-Cloud Program director, Chant is charged with implementation; he is - for now at least - the right man in the right place at the right time.
Microsoft man talks about UK.gov's cloud project
I put the question of whether the government has swung too far on cloud to Mark Ferrar, national technology officer (NTO) for Microsoft – which made the G-Cloud cut. Ferrar is intermediary between UK's public sector and the world's largest supplier of software.
As NTO, Ferrar acts as Microsoft's "trusted advisor" to senior politicians, civil servants and those running health trusts and local authorities. Ferrar's job is not to sell the next versions of Windows but to talk strategically about where Microsoft is headed.
Ferrar does bring a certain customer qualification to the position. Before joining Microsoft in October 2010, Ferrar ran the technology office in charge of NHS IT infrastructure. As a member of the UK government's chief technology officer council, Ferrar worked on elements standards on re-use and data that are forming the basis of G-cloud.
He was also involved in the national programme for IT at the NHS, a classic example of the type of big-box and centralised strategy that G-Cloud is supposed to kill. It cost £12.7bn and was axed for a decentralised strategy to re-coup money.
So, does Ferrar think the UK government has cloud fever? "I think it's been a bold step in the right direction to explore a technology that could have been unavailable to the public sector," Ferrar told The Reg.
"Is there too much cloud fever?," he responds. "The government will tell you they are not going to get it right on day one, they probably won't get it right 100 per cent and there will be a period of adjustment – and that's right and proper because the cloud and market will adapt," he said.
Ferrar said the government is in a no-win situation of staying with the past or moving on to a newer, more uncertain, future: "This is a seesaw moment... On the one hand you have focus and re-assurance of outcome but risk missing a shift in the way the market goes. On the other hand, you can embrace some of the vagaries of the market and you risk lurching from one side to another as the market finds its equilibrium. There's a danger when commentators say it's all going wrong or it's all going brilliantly well. We will see scepticism across the commentary [on] whether it's working or not and that's probably a healthy place to be."
With fever comes delusion, and tech companies are feeding the cloud fever by claiming their software, hardware or service is "cloud". We have three accepted definitions of cloud – IaaS, PaaS and SAAS – and plenty of scope to claim that one offers a product or strategy that "enables" them.
Try before you buy
As always, along with the hype comes the prospect of customers being oversold. As government customers stumble out of the shadow of centralised IT purchasing into the bright light of choice, what's to stop them getting ripped off?
Maxwell, government director of ICT futures, told me at the recent Intellect event in London that the government's terms and conditions would provide protection against this. Maxwell: "You test them out beforehand to see. They are getting accredited. Any proven IT director is going to test these out, government or non-government."
Ferrar agrees the phrase "cloud" has become overplayed and says he stands by the National Institute of Standards and Technology (NIST) definition from late 2011 here (PDF). He said that he thinks Microsoft plays into this with Office 365 and Windows Azure, which are listed on Cloudstore. "The cloud term is globally getting to point where nobody knows what it is but everybody kind of thinks what they know it might be," he says.
There is another challenge in the rush to the cloud, however: data. Right now, users around the world are blindly shovelling terabytes of data into different cloud services simply because it is so much easier to write, develop and deploy apps that run on servers that you don't need to manage. Amazon is the one everybody is chasing and it's a data sponge: the number of data objects held by EC2's S3 storage grew nearly 200 per cent in 2011 to 762 billion: 2011 was Amazon's fastest growth year.
A cloud provider will let you trickle in data in gigabytes or terabytes for months at a time; then you hit a year and then you have no capability to move. – Tom Hughes-Croucher, Joyent
But what if you decide you want that data back or want to change suppliers? The latter is a distinct possibility in government contracts, which have have fixed-length terms and where suppliers change.
It's easy to become trapped by a discussion about standards for portability – about the way cloud providers lock you in using data formats or APIs that don't interoperate with each other. According to Maxwell, the terms and conditions required to get on the G-Cloud supplier's list combined with "open standards" will solve the data lock-in problem.
But visit the Cabinet Office's current site on standards for data transfer in the G-Cloud, however, and you won't find much there to guide your purchasing decision.
In February the government started a round of consultations on open standards on data along with software interoperability and document formats. This was its second after the government controversially scrapped its first open-standards definition – apparently after it came under lobbying pressure from, ahem, Microsoft and the Business Software Alliance (BSA). The Cabinet Office, running the G-Cloud initiative, has denied this was the reason, however.
"We are at the beginning of it," Maxwell said of the new consultation. "We are saying we need to take control of the various components in your network infrastructure and end user devices, application development, service management and systems integration that now rests in departments."
The real problem isn't formats, though: it's sheer volume and completeness of data.
For example, neither Amazon and Microsoft charge a penny for putting your data into EC2 or Windows Azure, but going the other way, there's a toll. Amazon's exit charge starts at $0.15 per gigabyte and the firm offers discounts for volume. Microsoft's charges start at $0.12 per GB. Microsoft doesn't say how many objects are sitting in Azure but Amazon – as we know – is home to nearly 100 billion.
Tom Hughes-Croucher, chief evangelist at cloud software fabric specialist Joyent – one of those West Coast companies recently visited by MP Maude – told me that it is bandwidth that is the real choke point and that the UK government should be really concerned about. Hughes-Croucher was previously an evangelist and technical lead at web giant Yahoo!
"Getting the data isn't a big deal," Joyent's man told The Reg. "The big deal is a cloud provider will let you trickle in data in gigabytes or terabytes for months at a time; then you hit a year and then you have no capability to move the project because there's no bandwidth to move."
The last war
Hughes-Croucher's former Yahoo! colleague, Facebook performance and efficiency expert Carlos Bueno, tag-teams with him on this subject in this slideshow. Both agree that it's the network latency that's the real lock-in factor on cloud.
"Data formats are like fighting the last war. There are enough programmers that any weird binary XML could be reverse-engineered pretty quickly. That's not a big deal in my opinion. It's not strategic lock-in any more," Bueno says.
To give you an idea of scale: it took Yahoo! four weeks using 700Mbps pipes to move 10-20TB of data from servers run by Xoopit, which Yahoo! bought in 2009, to its own servers. Moving 12TB between Amazon S3 and Windows Azure will take 40 hours, and between S3 and Rackspace will take just under a week, according to this lock-in survey on GigaOm.
Clusters, clusters everywhere
The problem is that data centres cluster together in urban areas like London, and they don't communicate very well with their rivals over the road or in another part of the same country. Instead they link via fat pipes to their offices in other parts of the world such as Singapore, New York, or Texas. Cloud companies therefore can - and do - claim data portability is already solved because they can easily shunt data between their own data centres in zones around the world. This argument amplifies the point that moving data within the same cloud isn't a problem.
The solution is to have service providers interoperate with each other locally through fat pipes. Hughes-Croucher and Bueno say they think the UK government should use its size to demand that providers build cheap and fast data pipes between their buildings in the same area before putting their data in.
Maude: G-Cloud driver went West Coast
It's possible to get big fat, service-provider-to-service-provider connections, Hughes-Croucher says, but it's a case of having some incentive to do it. "Government is a big spender so whatever they say is their requirement has a big effect on the market," he says.
Yahoo! and Facebook, for example, have used their size to secure agreements with service providers for fast connections between their developer sites and data centres.
Microsoft's Ferrar reckons networks are improving and that this will help to move things around. "With fiber optic networks we will have the ability in the next few years to move more info almost instantaneously than has existed to this point. When we go to high-bandwidth IPv6-based networks between service providers and the telcos, the ability to move data between different parts of even our [own] cloud will be significant."
Once that happens, Ferrar reckons, the challenge will move from volume to whether or not the data that is uploaded is actually of any use. He says: "The question then comes: 'Do I understand what I'm putting into the cloud and I can I get it out in an understandable format?' Where we are today is: 'I had it here and I want to put it in the cloud and how do I do that?' We are seeing a lot of growth in BLOB storage, which is how a lot of stuff goes in."
BLOBs – or Binary Large Objects – are what happen when people are throwing around large pieces of data, typically at the start of a project or during a migration.
Microsoft's man reckons people need to get better at understanding what they are uploading to make sure it is really of use when the time comes.
"Drawing on my experience in health, one of the unrecognised positive outcomes of the national programme for IT was the understanding that data that goes into analysis and running the business for clinical and business outcomes has to be timely, accurate and complete – so they had a whole programme running on quality of data," Ferrar said.
"You need to train the organisation on how to manage data effectively. The health sector over the last seven years has learned to do that. Local and regional government will learn that as we move more data into the cloud [that] it has to be accurate and complete or that you have to know how inaccurate and incomplete it is. If you know that, then you've got a fighting chance of making some sense out of it."
Let's be clear on one thing: the new approach to government IT goes beyond a question. The European Services Strategy Unit in 2007 found examples of chronic failure in IT outsourcing with costs touching on £10bn. The idea of G-Cloud is an attempt to get away from IT mega-projects such as the scrapped £469m Firecontrol project.
The mantra from those driving G-Cloud is the involvement of small- and medium-sized tech companies. The understanding is that these SMBs can help effect change by being more nimble and competitive than the big boys in supplying government.
But what hope is there for tangible change when roster of existing, big suppliers crops up on the new Cloudstore? Cloudstore's approved suppliers feature IBM, Fujitsu, SAP, Dell, Rackspace, BT, CSC, Computacenter, EMC and Logica – not to mention Microsoft. Fujitsu and BT, for their part, are among those fingered by the ESSU for delays and overruns of projects under the old system. This bodes ill for both Maxwell's goal of help SMEs to "enter the government marketplace" and career civil servant Chant who's arguing for change on the back of years spent in different government CIO roles.
Worse still or a project with the word "cloud" in its name, none of these firms can really be considered a "flag-carrier" of cloud. That honour might go to industry leaders Salesforce, Google or Amazon, which are not represented in the Cloudsore.
Smaller cloud start-ups are missing, too. The UK government does use MongoDB, the NoSQL database supported by the 10Gen start-up Maude visited on his Californian trip. Yet 10Gen does not feature on the list of G-Cloud suppliers. In a statement, 10Gen technical director Alvin Richards told us that following an initial meeting with Maude to discuss the further use of MongoDB by the government his company will "engage in all necessary steps required to become an approved supplier."
Yet Microsoft is on the list. It's a company deeply embedded in UK government. Not just that, but Microsoft somehow convinced the UK gov that Cloudstore should run on its Windows Azure cloud. It was a neat trick, given Microsoft is not the first name that springs to mind when you mention "cloud" – many would say Amazon, Salesforce, Rackspace or VMware got their first.
Also, in terms of Azure at least, there's already official egg running down Redmond's face. On 29 February Azure was knocked offline for hours by a rather mundane, and easily avoided, leap-year programming glitch that dragged the Cloudstore down with it.
Given the UK government's huge reliance on Microsoft, isn't it a foregone conclusion that it should therefore go with Microsoft on G-Cloud in the same way it has gone with those other established, big suppliers of technology?
Ferrar puzzles at this and rejects the proposition. "It kind of implies the government is locked in [to Microsoft], when we don't feel they are," he says.
"I'm sat here 18 months into being Microsoft and you go back and look back and say: 'How did we end up like that?' and it's tough being asked to justify the success of a product that's been successfully marketed and does the right thing, for the right people, at the right time, and that allows them to collaborate in rich and varied ways."
One thing Ferrar does takes exception to is what he says is conflation between open standards and open source on the subject of cloud. "There are people trying to weave the two together, saying you can only have open standards if you have open source – and that's simply not true," he says.
Ferrar doesn't name names but insists there is a general level of discussion – some might call it a consensus – among developers working in the industry that the cloud will be built using open source. You have, for example, projects such as Open Stack trying to build a "Linux for the cloud" using a community model and released under an Apache licence. Amazon and Google, meanwhile, run on versions of Linux, not Windows – although Amazon does offer Windows instances for Redmond fans.
If you read the GPL, free means free as in freedom of speech, not free as in free beer. Stuff like that has cost of deployment and cost to patch it. – Mark Ferrar
"I'm unclear how or why these things need to be linked," Ferrar says. Proprietary products are still compliant with open standards."
He highlights Microsoft work that mixes the kinds of open-source being touted for the cloud with Microsoft's software: specifically putting Hadoop to crunch large batch jobs on Windows Azure and SQL Server. Before that, there was improved interoperability for MySQL and PHP on Windows and between Linux and Microsoft's Hyper-V, he said. "Microsoft has changed. Microsoft is a different animal to what it was five yeas ago," he says.
In combating open source, Ferrar talks total-cost-of ownership – a favoured weapon used by Microsoftees when they are trying to persuade you that Windows is better. The argument runs that support, training and maintenance come with a licence that is paid for – but that you will pay extra for such essentials when it comes to open source and Linux, whose purchase price is "free".
"We need to get away from this implicit concept that this is free – free 'no cost'," he says. "If you read the GPL, free means free as in freedom of speech not free as in free beer. Stuff like that has cost of deployment and cost to patch it. When you buy your Windows licence you get the cost to patch it."
Apparently this is what Ferrar told Francis Maude in July 2010 at a big CIO CTO council meeting, just before Ferrar jumped to Microsoft. Ferrar left the civil service for Microsoft in October that year.
Is Microsoft worried that the proposed change to government's IT to favour cloud services and SMBs will put Microsoft at a disadvantage? Microsoft, after all, built a 30-plus-year business selling licences for client and server operating systems, Office, email and collaboration software. Not according to Ferrar: "We embrace the change," he said. "We are embracing the change to open data."
But is Microsoft even ready for cloud, given February's outage?
We spoke to Ferrar before the leap-year crash. At that time, Ferrar reckoned the UK government had come to Microsoft – together with application integration partner Solidsoft – to put Cloudstore on Windows Azure because it "didn't want a fiasco with the thing rolling over."
What about now?
Following up afterwards, Ferrar pointed to the fact that Microsoft had been "open" about the cause of the outage. "We will strive to continue to be transparent with customers when incidents occur using the learning to advance our engineering, operations, communications and customer support and improve our service," he said.
It's early days for government cloud, both in the evolution of G-Cloud and the government actually taking its data and apps off existing servers.
Booby traps await, especially in data – where there are the rules that are supposed to govern this stuff alongside proposed standards that are works-in-progress. Despite this, the big-name tech vendors have already been signed up to G-Cloud. Their presence is bound to ensure standards that emerge suit their plans.
Microsoft's man has worked both sides of the public-sector street – on big, centralised projects as a customer and now as an IT supplier, somebody pushing cloud. Ferrar is optimistic: "Government would benefit from a diving towards greater shared agenda – sharing solutions," he says.
And that, it would seem, means cloud. ®