Ceph vs Swift for OpenStack object storage, why the ‘pros vs cons’ approach to evaluation is a flawed analysis

For a casual outside observer there’s a lot in common between Ceph and Swift: they are both open source projects, they have both enjoyed major and ongoing increases in the number of developers actively engaged in improving them, they are both mature, and they both have a legion of fans with serious engineering skills and live deployment experience. Each camp extolls the virtues of their preferred approach, and acts as cheerleaders encouraging its adoption. Supporting either has to be viewed as a win for the Open Source community overall.

Ceph – if you can forgive the pun – was out of the blocks first in this two-horse race, launching in 2006. Swift launched two years later in 2008, and has been playing catch up ever since. Ceph delivers unified storage, supporting File, Block and Object. Swift is Object only. Ceph is an independent open source project. Swift was originally part of the Open Stack project – though the company that owns it, SwiftStack – is moving it on from this heritage. The general consensus is that Ceph is something of a ‘jack of all trades’, complete with the accompanying inference of ‘master of none’, whereas Swift does one thing well, but one thing only – giving it the polar opposite of inferences – that of the ‘one trick pony’ – SwiftStack are working on file-based services, they haven’t arrived yet. So, when it comes to the speciality of Swift – surely the choice is obvious, the specialist in Object, and part of OpenStack, and therefore the best choice when looking at this configuration?

Well no, not really, its not that simple. There are two strong reasons to prefer Ceph to Swift – reasons which those legions of fans (on both sides) overlook, because they have pretty much nothing to do with engineering virtues and everything to do with human behaviour, the efficient use of skilled engineering resources, and support contract cost management in the enterprise. Before I get to that, let’s take a shallowish dive into the major differences – just for the sake of form.

In favour of Ceph.

• The obvious point of File, Block and Object in the same wrapper. It might be an obvious point, but it’s a pretty damn important one.
• Better transfer speed and lower latency – because traffic to and from the Swift cluster goes through proxy servers which slow it down.
• Swift can have further latency problems as replicas are not necessarily updated at the same time, so requesters retrieving data can access old – wrong/outdated – versions.

Against Ceph

• Ceph’s multi-region support – usually touted as an advantage, is in a master slave configuration, but as replication is only possible from master to slave, in a deployment with 2+ regions you can get uneven load distribution. Not a problem in Swift.
• In Ceph you should only write to the master. . . but there is nothing to stop you from writing to the slave which can mean poor execution resulting in inconsistencies and, in extreme circumstances, complete corruption. Not a problem in Swift.
• There can also be a security issue as RADOS clients on the Cloud compute node communicate directly with the RADOS servers over the same network Ceph uses for unencrypted replication traffic. So, potentially, if Ceph client node is compromised, the attacker can see all traffic on the storage network. Not a problem in Swift.

The security problem is a bit of a straw man as best practise demands a separate network, and in any case I’m knit picking the problems – working hard to find the cons.

But, as per the title, none of these pros and cons are relevant, and in any case, as both approaches can work alongside each other comfortably, should you be making an ‘either or choice’ in the first place? Well, as I said earlier, there are two concrete reasons why Ceph is the winning approach.

#1. Because running open source technology requires skilled engineers.

Anybody in the proprietary camp will tell you that the money you save by avoiding software costs can come back in additional engineering skills costs: paying for the support contracts or skilled headcount required, and keeping that skilled headcount up to speed with developments comes at a cost. Just how many different skill sets can you actually master? When there are two different ways of doing an open source approach smart enterprises will adopt the tech which makes this headache as small as possible.

#2. Because Swift is busy working on proprietary APIs that not only differ from Ceph, but also from Amazon Simple Storage System, which will potentially lead to widespread resistance to ‘yet another storage interface’.

In reality the choice is simple, albeit uncomfortable for enterprises and individuals who have invested a lot of time and resource into getting good at Swift. Ceph is a Swiss army knife, complete with the Swiss army knife’s array of potential use cases: corkscrew, screwdriver, saw, bottle opener, even a needle. Meanwhile Swift is a really great pen knife. Who cares if the blade is sharper? When you’re in the shop getting ready for the camping trip, who even checks?

Who can rationally choose the lower number of use cases? Don’t ask the fans – the support of fans is simply not rational.