PayPal steps it up with OpenStack

Soumik Ghosh September 30, 2015
PayPal steps it up with OpenStack

Payment pro PayPal recently migrated from VMware to OpenStack. Sri Shivananda, the man who spearheaded the migration talks about the rationale behind the move, and shares his two cents' worth for CIOs contemplating a cloud switch.

Payment pro PayPal recently migrated from VMware to OpenStack. Sri Shivananda, the man who spearheaded the migration talks about the rationale behind the move, and shares his two cents' worth for CIOs contemplating a cloud switch.

PayPal converted nearly 100 percent of its traffic serving Web/API applications and mid-tier services to run on its internal private cloud, based on OpenStack. The prime mover behind this migration, Sri Shivananda, in a career spanning 18 years, has been featured in Silicon Valley’s ‘Top 40 under 40’.

It’s a common misconception that OpenStack is a hypervisor.  In truth, OpenStack is a manager of multiple hypervisors. OpenStack sits above pools of virtualized resource as a master control plane that provides users with a single point of control and orchestration.

PayPal developed its own OpenStack deployment and started a transition away from VMware, mainly because of the benefits of being able to customize its own platform – Something that is out of reach for the majority of IT organizations.

Sri spearheaded the move, making PayPal’s OpenStack clouds one of the largest financial services clouds in production today.

“What OpenStack provided us was a holistic operating system, which we could do virtualization on, and also manage how we onboard customers,” explains Sri Shivananda, Vice President, Global Platform and Infrastructure, PayPal.

The journey took about two and a half years to complete. Last year, PayPal attained a 100 percent OpenStack cloud for the front and mid-tier.

So, why OpenStack?

OpenStack gave PayPal more control of its infrastructure. When you run a large scale internet site like PayPal, there’s a lot at play, including scale.

Orchestrating that virtualization and creating a fleet that is usable by its internal customers is what OpenStack provided PayPal with – A holistic combination of what’s needed for compute, network, and storage.

So, in a way one should think of it as an evolution, and it’s the next step in the journey most enterprises take on their private clouds.          

Teething trouble

OpenStack is still a maturing system, and many people who are running it out of the box, run into issues in their environments, availability challenges, and so on. “We’ve taken a more holistic approach, and built layers that create reliability and availability,” he says.

So, if a primary part of the system actually fails, it can switch over to a secondary, in turn managing the traffic. Therefore, redundancy and resiliency are the two investments PayPal made on top of OpenStack.

Shivananda believes Openstack is like a teenager – Still just growing up. There is a lot of promise and potential in it. “We’ve been able to leverage that over the last two years, and we continue to do so.”

But, there a lot of things that still have to be figured out. The next version, called ‘Kilo’ actually is a storehouse of features. It’s becoming a more holistic ecosystem that truly delivers the capabilities that’s needed in compute network storage.

OpenStack is probably enterprise-ready, but being web-scale ready is different. There are perhaps two dozen companies in the world that operate it on really large scale. To be ready for that, it needs more maturity.

As the community expanded, some of the developmental pace has reduced on Openstack. “This where we continue to emerge in these technologies. While we have a very robust Openstack implementation on the PayPal side, we are now beginning to look into the next generation of technologies,” says Shivananda.

PayPal still continues to use Openstack, but is beginning to operate on what is now being termed as the Data Centre OS: with Mesos, Kubernete, and Docker.

The open community

One of the biggest reasons PayPal moved to OpenStack was that instead of building proprietary software to manage its cloud, it now had an open community it could collaborate with.

This is in fact a great way to innovate together, and solving industry problems in a consistent way. So, one day, when the company may want to move to a public cloud, the transition will become seamless. 

There are hundreds of vendors participating in the OpenStack ecosystem. Among other players, NASA is one of the forerunners in the community, and RackSpace has a big OpenStack contingent.

The cloud clash: Public, Private or Hybrid?

One size doesn’t fit all. PayPal built a private cloud because the scale is very large, and security is paramount. The company, therefore wants to ensure that it has full control of the fleet that it has built. It has various mitigations in place to ensure that it is secure, and is providing what is needed for customers in terms of safety.

As the economics of the public cloud is significantly changing it’s becoming more and more affordable. The pay-as-you-go model is a thing of the past.

For CIOs migrating to the cloud:

It all boils down to the adoption of cloud principles. More often than not, organizations make the mistake of thinking that migrating to a cloud is a ‘lift-and-shift’ operation. It’s not quite that. In many cases, it means having to re-architect the application to be cloud-ready and then the migration can begin.

“My advice to folks would be that you have to start small. You conduct a proof of concept, take a single application, make it cloud-ready, and get it on a cloud environment. Then, over a period of time, you move the rest of the applications to the cloud environment,” believes Shivananda.

In a cloud world, the architecture has to be infrastructure-agnostic. The application has to adjust to that infrastructure. In the past, the application would drive the hardware, but in the cloud world, you can’t have the hardware being hardware-specific. It has to be infrastructure-agnostic.

The Big Bang approach, and why it doesn’t work

The prime reason is that every cloud journey is unique. “I’ve been a part of two now – E-bay and PayPal. I’ve seen that there are various parts of the internal architecture that play a role in how that migration works. So, it’s an iterative learning process. It’s not a switch you flip and it happens,” explains Shivavnanda.

The risk of a big bang approach is extremely high, both in terms of impact to the customer, and what meets the success of the program itself. “You’ll see that a team builds confidence through a small success, and then they are ready for something larger,” says Shivananda.

PayPal’s governance model

PayPal doesn’t have different teams in the organization building infrastructure. It centralizes, homogenizes, and creates one offering. That way, PayPal takes all the principles of architecture, and pushes it down into the frameworks. And so, not every person who’s using it needs to comply with something, it’s automatically enforced for them at the infrastructure layer. In most cases, governance is being done through check points and people processing.

Now, that ends up being bureaucratic, or blockers to innovation. “We want our engineers to be artists, and not mechanics,” concludes Shivananda.  

LATEST CASE STUDY