When I joined IBM in 1994 it was as a manager of a project to create an executable model of the IBM business processes. This would be used to prototype process changes and potentially create reusable business objects for future applications. We got started in September of '94 and somehow got on the calendar of the IBM CIO to deliver a demo in February of '95. There was a lot of code to sling and not much time. A critical piece of the demo was being developed by another new hire who happened to catch the flu but worked very long hours with a high fever and delivered all that we had asked for. That was how I met Toufic Boubez. Since then he has done a lot of work in the area of Services Oriented Architecture including writing a couple of books, participating in standards bodies, and co-founding some software companies.
John - Toufic thanks for taking the time to speak with me today. Before we get started on the core interview questions I wonder if you could tell me what you are currently working on.
Toufic - I have co-founded Metafor Software. We are developing next generation tools for the data center. We started with the cloud. Our original focus was how do we help companies who already have applications running, in moving them to the cloud. The more we dug into that problem the more we figured out that it was a much bigger problem. It was about application mobility and migration, even if you are not planning on moving to the cloud. So we are building next generation tools for the data center, where cloud migration is a special case of what we will offer. The first generation tools have all been infrastructure centric, telling you about your servers, your load balancers, your networks, all that kind of stuff. Buts that not enough. You don't have visibility at the application level. What are the applications?, how many instances are running?, is it distributed over different servers? What if I want to move an application? What will be the impact?
John - Cool. How do you tag the application instance so you know it on a specific server and is sending the traffic you are monitoring?
Toufic - You hit right away on the crux of the matter. There are two really hard problems involved with this. One is discovering the components that make up the application, and the second is aggregating them into an application. So the automated discovery is a big issue. How do you discover that you have Apache server running?, you have Ruby running?, you have PHP?, you have SQLdatabase?, all that kind of stuff. How do you figure out the configuration, the boundaries, memory requirements and other dependencies? That is the first order problem. Once you have that figured out, how do you identify an application where these components are distributed across several servers? And there are a few ways to do this. I am not going to give away the secret sauce of Metafor yet. But once you have identified a model for an application, that may have 0-n instances of concrete applications. How do you take one of these currently running on three servers and drag and drop it onto some other environment... other servers in the data center or maybe onto an Amazon cloud?
John - Sounds fairly leading edge. Have you got some deployments with customers yet?
Toufic - We are still in development mode with some pre-subscribed customers waiting for an Alpha release sometime this Spring.
John - So what have you seen as a pattern of success with companies that have adopted SOA?
Toufic - There is one thing that keeps coming up over and over. It is essential to get management on board the initiative. You have to have buy-in from the management... it is not just a technology thing. With SOA we need collaboration between IT and Management, with both parties on board.
John - What does it mean to be "on board"? What are some of the concrete measures that would prove that management were on board?
Toufic - One big one, is that with an SOA adoption, the first project will always take longer and cost more then the legacy way of development. This is where you separate lip service from actual adoption. When a manager has to commit a budget increase or schedule delay on a project in order for it to be SOA.
John - How many projects will they have to invest in before the ROI curve beaks even?
Toufic - It will take at least a couple and maybe more depending on the company. One factor is skills of the developers, they have to learn how to design a services oriented architecture. And, second, you have to develop a library of reusable services for the next projects to benefit from, before savings are realized.
John - If you have a team that has learned to develop an object oriented system with reusable objects will there adoption of SOA be more straight forward?
Toufic - Yes it will. I believe that there is a two thirds overlap between the skills used in OO and in SOA.
John - Any other patterns of success?
Toufic - Yes, the second one is governance. The concept that up front you are not building some web services willy nilly, but you have a strategy, have a set of guidelines that will be used throughout the organization. If you don't set up governance then you will have web services technically, but not from an enterprise point of view.
John - In a larger IT shop how heavy handed should the governance be? I can imagine in a small company the governance could be relaxed but in a big company it would need more bureaucratic standards.
Toufic - The real key to success is to have the governance program fit the existing culture of the organization. Whatever the size of the company, figure out how formal they already are around other processes and make the governance fit that pattern. For example, the registry of services. I have seen some companies be very successful going with a full UDDI while others of the same size are just as successful in using a Wiki.
John - Lets move on to failures. Give me a couple of war stories around companies that have failed in adopting SOA.
Toufic - There is one that really sticks in my mind. In this organization the management was on board and some of the developers. But there were also a number of developers who were very passive aggressive and dragged their feet, and just wanted to see SOA fail. There was no communication... That reminds me of a third pattern for success. In order for SOA to catch on the leadership in the company have to communicate and educate on the strategic importance of why SOA is important to the success of the organization.
In this particular organization we had the passive aggressive behavior and the other real mistake was to try to analyze the entire organization and implement all the web services in a single massive effort. They spent almost three years on just the analysis/design phase of the project. They need to do it on a project by project basis. That way the organization will begin to see the benefits from SOA much more quickly.
John - Kind of reminds me of when we met at IBM on the BCOOL project, trying to model all of IBM's business.
Toufic - Exactly. Again and again through all the technology cycles we have been through we see the same patterns. Just as an aside, every wave of technology we think will be rapidly adopted and will immediately revolutionize the IT world. But it just does not happen like that. Its been what? Ten years since web services were first introduced and it is just starting to be really widely adopted.
John - So how large has SOA become? What percentage of new applications do you think are based on web services?
Toufic - I don't have any market data, but I can tell you that there is not an industry segment where some adoption is not taking place. And we have to include REST in the discussion along with SOAP and SOA and web services. If you combine these together then I think it is a large percentage of new apps.
John - Any other failures?
Toufic - Yes. I have seen a lot of shops build web services in a silo with no enterprise collaboration. They have lots of web services based on the technology of SOA without any of the real economic benefits of reuse.
John - What do you think THE NEXT BIG THING will be in SOA?
Toufic - In the next three years I think we will see the blurring of the lines of where things live. Right now we are falling in the trap of building silos again. We are creating silos based on what data center they live in or if they are on an Amazon cloud or an IBM cloud. We need applications to be able to move freely and transparently between these environments.
John - When you talk about the infrastructure in the data center, are we not talking about the OS and the database and the middleware etc? And is not that same set of infrastructure components available in a public cloud?
Toufic - Yes, but the management interfaces are different and know only of the applications in their domain. And there are different security management styles in place. Today there are separate boundaries around each of these environments.
John - Would an intermediate step be for a vendor like a Microsoft who has a cloud product and a data center OS / DB be able to offer a vendor vertical solution whereby they would allow a correctly configured application to be managed from a single console and moved between data center and cloud with little impact?
Toufic - Microsoft is a one vendor environment with lots of controls over that environment. And as long as you adhere to the Microsoft way then the transparency between data center and cloud will exist, BUT if your company wants to work with Java, PHP, Ruby etc then it would not work so well.
John - So to get to the federated cloud environment all the big players will have to agree on some standard for all these interoperability details?
Toufic - Well that is one way. There are a couple of other ways to do that.One is to encapsulate the run-times and the other is to create models that can be rendered on the different platforms. Frankly, I think that agreement on standards would be the most difficult of the three to achieve.
John - Thanks a lot for giving us your insights what has worked and not in the world of SOA.
John - If you have a team that has learned to develop an object oriented system with reusable objects will there adoption of SOA be more straight forward?
Toufic - Yes it will. I believe that there is a two thirds overlap between the skills used in OO and in SOA.
John - Any other patterns of success?
Toufic - Yes, the second one is governance. The concept that up front you are not building some web services willy nilly, but you have a strategy, have a set of guidelines that will be used throughout the organization. If you don't set up governance then you will have web services technically, but not from an enterprise point of view.
John - In a larger IT shop how heavy handed should the governance be? I can imagine in a small company the governance could be relaxed but in a big company it would need more bureaucratic standards.
Toufic - The real key to success is to have the governance program fit the existing culture of the organization. Whatever the size of the company, figure out how formal they already are around other processes and make the governance fit that pattern. For example, the registry of services. I have seen some companies be very successful going with a full UDDI while others of the same size are just as successful in using a Wiki.
John - Lets move on to failures. Give me a couple of war stories around companies that have failed in adopting SOA.
Toufic - There is one that really sticks in my mind. In this organization the management was on board and some of the developers. But there were also a number of developers who were very passive aggressive and dragged their feet, and just wanted to see SOA fail. There was no communication... That reminds me of a third pattern for success. In order for SOA to catch on the leadership in the company have to communicate and educate on the strategic importance of why SOA is important to the success of the organization.
In this particular organization we had the passive aggressive behavior and the other real mistake was to try to analyze the entire organization and implement all the web services in a single massive effort. They spent almost three years on just the analysis/design phase of the project. They need to do it on a project by project basis. That way the organization will begin to see the benefits from SOA much more quickly.
John - Kind of reminds me of when we met at IBM on the BCOOL project, trying to model all of IBM's business.
Toufic - Exactly. Again and again through all the technology cycles we have been through we see the same patterns. Just as an aside, every wave of technology we think will be rapidly adopted and will immediately revolutionize the IT world. But it just does not happen like that. Its been what? Ten years since web services were first introduced and it is just starting to be really widely adopted.
John - So how large has SOA become? What percentage of new applications do you think are based on web services?
Toufic - I don't have any market data, but I can tell you that there is not an industry segment where some adoption is not taking place. And we have to include REST in the discussion along with SOAP and SOA and web services. If you combine these together then I think it is a large percentage of new apps.
John - Any other failures?
Toufic - Yes. I have seen a lot of shops build web services in a silo with no enterprise collaboration. They have lots of web services based on the technology of SOA without any of the real economic benefits of reuse.
John - What do you think THE NEXT BIG THING will be in SOA?
Toufic - In the next three years I think we will see the blurring of the lines of where things live. Right now we are falling in the trap of building silos again. We are creating silos based on what data center they live in or if they are on an Amazon cloud or an IBM cloud. We need applications to be able to move freely and transparently between these environments.
John - When you talk about the infrastructure in the data center, are we not talking about the OS and the database and the middleware etc? And is not that same set of infrastructure components available in a public cloud?
Toufic - Yes, but the management interfaces are different and know only of the applications in their domain. And there are different security management styles in place. Today there are separate boundaries around each of these environments.
John - Would an intermediate step be for a vendor like a Microsoft who has a cloud product and a data center OS / DB be able to offer a vendor vertical solution whereby they would allow a correctly configured application to be managed from a single console and moved between data center and cloud with little impact?
Toufic - Microsoft is a one vendor environment with lots of controls over that environment. And as long as you adhere to the Microsoft way then the transparency between data center and cloud will exist, BUT if your company wants to work with Java, PHP, Ruby etc then it would not work so well.
John - So to get to the federated cloud environment all the big players will have to agree on some standard for all these interoperability details?
Toufic - Well that is one way. There are a couple of other ways to do that.One is to encapsulate the run-times and the other is to create models that can be rendered on the different platforms. Frankly, I think that agreement on standards would be the most difficult of the three to achieve.
John - Thanks a lot for giving us your insights what has worked and not in the world of SOA.
Thank you for the info. It sounds pretty user friendly. I guess I’ll pick one up for fun. thank u.
ReplyDeleteFunction Point Estimation Training