Archive for the 'Software' Category

« Previous Entries
Next Entries »

SOA governance largely not established

Saturday, October 6th, 2007

The SOA Forum surveyed over 500 organizations and concluded that nine out of ten haven’t figured out SOA governance.  I’m quoting it here because this find actually matches my experience with SOA initiatives I’m seeing in financial services IT organizations.  The banks that have advanced their SOA initiatives certainly have some form of governance, but it’s overwhelmingly manual and, for the most part, cover only some areas.  In those organizations, there are some impressive metrics, but they are compiled manually, and quality and policies are implemented casually (or not implemented at all).  The IT organizations I deal with have multi-billion dollar IT budgets and thousands of employees, so formalized and automated governance is going to be key at some point, or it simply won’t scale, and there is a high degree of risk the SOA will fail.

Some more color is needed here though, because SOA governance means different things to different people.  Many view policy enforcement as the sole mechanism for governance, and although a managed policy structure with automation is indeed and very relevant and important, there are other quite necessary functions in the governance realm as well, and few IT shops have implemented them.  For instance, successful SOA projects have established a common support framework mapped to their process including automation supporting the following:

  • Requirements capture and management
  • Project management
  • Ticketing management
  • Collaboration between teams around common services
  • Registration of services and the certification of the advertised state of each
  • Discovery of services for orchestration of composite solutions
  • Consumption of services in development, testing and production
  • Metering and billing of components in the runtime environment

Clearly, automated policy management can “check” things going in and out of registries and a few other places but a complete approach to governance is rarely seen implemented, (meaning a unified approach to cover the project management activities, runtime metering, bugs recorded, collaborative activities surrounding moving components forward or integration strategies, etc).  The convergence of these things in an automation paradigm coupled with a federated process to ensure teams are on the same page makes up the governance strategy of a successful SOA.

The survey referred to above also mentions that 56% of the respondents indicated that more than half of the software artifacts developed in-house are not reviewed for compliance before moving into production.  This problem isn’t going to be solved with a merely a policy manager.  Unless governance reaches the various functions and activities that touch a component, there will always be ways in which policies can slip through the cracks.  SOA governance needs a very broad, horizontal approach, one that maps process to support infrastructure covering the bulleted items I’ve listed above.

Although the survey asked the respondents for an estimate of how many services are checked when automated policy management is in place (to which the survey reports 88% say more than half of their services are checked, and 50% indicated all are checked), the question I have to ask is: what is “policy management” in the minds of the respondents?  There isn’t a clear answer here, and based upon my interactions with IT organizations in large financial institutions, I tend to believe there are varying ideas out there with respect to the definition of policy enforcement.

Other survey’s I’ve seen report very low percentages regarding satisfaction with governance solutions, but trend upwards for those with automation solutions.  Relatively speaking, it’s safe to say that automation supporting governance processes is key, and the broader the automation covers the more beneficial the governance is to the business.

Posted in Software, Trends | No Comments »

SOA, a Business Facilitator

Thursday, September 27th, 2007

This might sound shocking coming from a CTO, but SOA isn’t about technology, it’s about facilitating business.  Of course all of IT could be viewed that way.  I go deep into SOA projects in financial institutions on a regular basis and I find many of them initiated and driven out of IT, but with little initiative, support, even awareness from the business of SOA.  Most of the time it boils down to mere education, but whatever the case if an SOA project is purely IT, there will likely be a struggle.

I see metrics reported like “components reused” (don’t get me wrong, that’s a good metric, I report on that metric for SunGard internally as well) but there needs to be business context around it in order for the business to comprehend the value (for example, I include revenue generated, projected, etc. for each component, as well as some subjective attributes like revenue that wouldn’t have otherwise happened without SOA).

Last week I was with a customer whose SOA project is in trouble.  My interpretation of their situation is very simple… IT has little to no involvement from the business in their SOA effort.  They took the approach that the SOA “technology” could be inserted at a layer beneath what the business sees (i.e. the apps and functionality mapped to the business doesn’t change, and the SOA is inserted beneath, abstracted away).  Requirements are written almost exclusively by system analysts and no benefits were exposed to the business.  The project started failing when the business noticed service interruptions and other unexpected problems.  That’s completely the wrong way to go about it.

I routinely state that the process is more important than the technology, and here I’m throwing business direction, support and even leadership into that category as well.  The successful SOA’s have a core team working in a federated way with each business lead and their teams in an organized and collaborative manner, taking direction from the business and exposing value “chunks” little-by-little along the way.

Posted in Software, Trends | 2 Comments »

SOA, SaaS, BPM and Open Source and Better Composite Solutions

Friday, August 3rd, 2007

If ever there were four completely independent concepts (in terms of evolution) that are destined to be related (even in the immediately family) and together revolutionize our thinking about how software delivers solutions to meet agile needs business needs, it is SOA, SaaS, BPM and Open Source. 

For a couple of decades people have made attempts at componentizing software for the purposes of reuse and rapid composition of solutions with certain measures of success.  Overall, the componentizing of software into parts that can be reused and assembled into composite solutions is far from an exact science. 

Many years ago I served as CTO for a hardware company that produced parallel processing chips for the super-computing marketplace (supplied to NASA, DoD, various government contractors, etc…).  I still remember battles my electrical engineering staff would have with my software developers regarding the lack of exactness in the componentization of technologies and the assembling into the solution (our product).  Developing hardware takes a very systematic approach (real engineering) involving libraries of components and assembling them together (via VHDL or schematic capture, for instance) into the ultimate solution.  Simulators can tell you if you get what you expected far in advance of production.  The components follow exact interfaces, the libraries are clear and open and the simulator works in conjunction with development environment to provide a complete view of what’s happening.

When the time would come to write the driver and supporting software for the hardware, the battles would ensue.  It became apparent that although I had been developing software for many years, I simply didn’t realize how non-exact (un-engineering-like) software development really is.  A driver and other software required didn’t have the luxury of open libraries to pick from, simulators to run exact assemblies through for precise results, instead the software was mostly not exactly what you expected, until multiple iterations of development and testing.  When the hardware is compiled, you have one shot at getting it right, whereas with software each compile yields an attempt at getting closer to what you want.

After multiple attempts at defining distributed component technologies for software (CORBA, DCOM, EJB, etc).  I think we’re finally getting very close to a far better component-and-assembly world for software solutions with the combined power of SOA, SaaS, BPM and Open Source.  SOA yields better standards than we’ve ever had before (not just in terms of software interfaces including WSDL, BPEL, SCA, etc., but the collaborative governance around how an enterprise functions as well), SaaS for the on-demand environment, BPM for the modeling and assembly of the solutions and Open Source for the openness of libraries and infrastructure. 

Having lived the differences between what the presence and absence of these attributes can have in the development of solutions with technology, I’m convinced these are powerful (and extremely related) attributes to successful composition of solutions from reusable components.

Posted in Software, Trends | 1 Comment »

Vertical SOA

Tuesday, July 17th, 2007

There is a lot of attention currently about SOA offerings going vertical (check out IBM’s announcement here).  We’ve been saying for years that the general everything-to-everybody nature of SOA has to get more specific to the business domain in order to address many of the challenges companies face in SOA implementations.  Recently, SunGard announced Infinity, a SaaS platform that combines BPM and SOA in an on-demand offering exposing SunGard software assets in an agile composition framework for solution delivery.  Infinity was built with Financial Services in mind, very vertical focused.

It shouldn’t be a surprise that SOA is going vertical, because a lot of horizontal technologies end up with specific industry focus, such as how EAI and EII eventually packaged adaptors for specific domains, or how BPM is doing the same (you see HIPPA specific support for health care applications and SWIFT, FIX, etc. for Financial Services, and so forth).

We’ve introduced Infinity as unique in terms of on-demand delivery with the agility of SOA and the needs-modeling of BPM, surrounded by our Financial Services expertise.  From our discussions with our customers, we’re finding the vertical focus is resonating.

Posted in Software, Trends | No Comments »

SOA Soup

Tuesday, May 8th, 2007

As a follow on regarding my blog on Financial Services SOA Adoption Challenges continued analysis from many is adding more color to the gotchas.  CIO magazine ran an article in the May 1, 2007 issue entitled “Stuck in the SOA Soup”.  The article points out there are 115 standards related to SOA (quoting Forester Research).  The plethora of standards means there is no unified approach to SOA (not to mention process).  There’s a simple but spot on quote in the article from Hong Zhang, director and chief architect of IT Architectures and Standards at General Motors: “The challenge is that there is no common, consistent architectural framework to guide the evolution to SOA”.

I see this problem rearing its head in Financial Systems IT shops.  Most are defining their approach from scratch, and this leads to lots of learning (making mistakes, correcting them, etc. some are successfully maturing their approaches gradually, but it’s taking a lot of time, while others fail altogether).

The article suggests focusing on quality of service (I fully agree, see my #1 point in my blog entry The Registry is Important but There’s Much More).  Also, the article highlights General Motor’s experience.  It’s a good read, I’d recommend it.  Interesting though, the article says GM launched their SOA in 2000, and back then there was far less definition around framework, in fact the term SOA didn’t even exist, (we launched our SOA effort in 2002, and it didn’t exist then either, we merely began with decomposing apps into loosely-coupled common services, reuse, collaboration and a governance process… all of which became molded into the SOA buzz).  The article’s main point (which I also fully agree with) has to do with achieving agility.  Quoting Mr. Zhang again regarding the goal of an SOA is “to establish a flexible information systems and services environment that can quickly realign” as business needs change.  Absolutely correct.

Posted in Software, Trends | No Comments »

Financial Services SOA Adoption Challenges

Wednesday, May 2nd, 2007

This week in the April 30 InformationWeek publication there was short couple of paragraphs under the heading “SOA: A Warning Shot”.  The short article reports that financial services firms that have implemented SOA’s are “… finding that they can be more expensive, less reliable, and harder to maintain than standalone applications” (quoting Sajay Sethunath, chief architect for BearingPoint’s financial services consulting business.  From the article:

“His conclusions should be a warning to other industries.  IT departments tend to pick the low-hanging fruit and build services that they know will be reused by several applications, he says.  But too often they haven’t established a metric to show how the effort leads to savings.  What’s worse, Sethunath says, many of the services aren’t even what business users want.”

I work with financial services firms every day, and know many CIO’s and CTO’s in the industry personally.  I can say with confidence, there is truth to Sajay’s comments.  Some are indeed struggling with early implementations of SOA’s. I know of IT organizations in financial services that have made meaningful progress, although many are in early stages without the skills and knowledge to build a complete SOA infrastructure. 

Research done by the Aberdeen Group says that cost-savings remains the biggest lure of SOA.  But regardless of where the IT organization is in the SOA implementation, the big payoff though is a ways out.  The analyst at the Aberdeen Group gathered data from over a thousand companies, and the report basically concluded that if 2005 was the year of SOA romance, the honeymoon was over by the end of 2006, in other words reality is setting in.

SOA’s are valid and worthwhile for any financial services IT organization, but the reality is that they require a systemic change in architecture and process. SOA’s are very complex.  After seeing successes and failures win lots of organizations, many of the key glitches are becoming clear, they are:

  • Don’t underestimate the power and usefulness of the right tools, like a flexible and extensible registry to manage services, policy servers and collaboration wikis.
  • Quality, quality, quality.  Because distributed components loose the benefit of an all-seeing and all-knowing compiler, there isn’t a quick “syntax error” message you’ll get when a disparate service is consumed for the first time in a unique composite application.  The registration process of any service needs to include comprehensive testing and certification.
  • Follow popular standards and create your own where they don’t exist.
  • Pick the right people, the right skill sets.  People that can see and understand the end from the beginning, but can also manage daily tactics of what needs to be done to get there, (Steven R. Covey’s famous book “7 Lessons of Highly Successful People” defines who the right people are to build an SOA as far as I’m concerned).
  • Understand and define what business issues exist that can benefit from the SOA.  Keep that definition in mind throughout the process, in particular when things start to get off track.
  • Set realistic expectations.  The big payoff may take a while to realize, so the value needs to be understood in a temporal context.

No pain, no gain.  Nothing worthwhile is every easy, so don’t lose heart.  It’s the right architecture and process that can today’s business challenges.  Think of it as a journey, not a destination.

Posted in Software, Trends | 5 Comments »

SOA Decomposition

Monday, April 23rd, 2007

At any given time, SunGard has hundreds of active projects involving the decomposition and evolution of its systems, legacy and contemporary alike.  We track and monitor these activities as an organization because many of them are collaborative in nature.  Our SOA effort (the SunGard CSA) enables the collaboration by ensuring reusability of work done, common standards, vision and strategy is achieved over time and an overall customer experience is controlled and enhanced.

I’m often asked how legacy decomposition is going at SunGard.  One customer even used the word “frightening” in a recent conversation where the size of the SunGard portfolio of applications was apparent in the dialogue, and he was curious if the sheer size of this effort would be adding inertia, such that the weight of each situation adds resistance to the whole, eventually slowing it down to a dead stop.  I was happy to report that due to the federated (decoupled and distributed) nature of the SunGard SOA effort (CSA) we’ve found that each project isn’t added weight, its parallel (weight none the less, just not additive), so even though a mammoth undertaking like this looks “frightening” on the surface, it’s actually quite systematic.  Our experience has been one of acceleration with each collaborative activity, rather than burdensome added weight.

I also shared several key attributes we’ve learned over the years to assist with educating people who are tasked with the daily architecture and business process work that surrounding a decomposition effort.  I’ll include in this blog the key success factors to the process of decomposition that I shared in the conversation for everyone’s benefit.

First, I shared that recognizing resistance patterns in teams and turning them into adoption patterns for success is the first thing to establish.  I’m working on a white paper at the moment that articulates this concept, so I’ll leave that topic alone here, and will make a blog entry when the paper is available.

Second, starting a project where refactoring and/or decomposition will be used for the purpose of evolving an application into an agile and flexible system of configurable components or services requires a new way of thinking.  I face this issue continually at SunGard, i.e. smart people that have been maintaining and enhancing the same systems for years, and getting people to think differently when their daily lives are so engrained in a particular methodology can be challenging.

In a nutshell, if the education is founded in sound principles, usually the teams catch on quickly.  The method we use includes methods for both business processes (containing units of business functionality) as well as data (providing context for the process and functionality of the parts).  Both have to be modeled in their current form (the initial take should be extremely high-level), meaning a sheet of paper should contain a flow diagram of the process and another sheet containing the data and its relationships.  Keeping the first draft very high-level (almost trivial) will assist in ensuring the team slowly gets converted to the SOA-way of thinking, because the cold-turkey method doesn’t work well with teams that have been working with an application for years.

From the high-level process and data diagrams, iterations will start to decompose the components further and further, until the lowest-level of the business process and data model have been documented.  With each iteration, education is required to ensure the teams are contributing and evolving along with the process. 

At that point, the more interesting phase begins, where the list of decomposed parts are then architected in component form (this is where the determination of re-writing, combining or refactoring any of the resulting parts is determined versus merely exposing and interface of an existing component).  This is also where the SOA begins to take shape, such as loose coupling of the parts, decisions on protocols, interfaces, events handling and other infrastructure as well as process modeling with the case studies.  In a parallel effort to the iterations, case studies that hold examples of the desired end-result should be considered and tested intellectually.

There are several keys to this stage as well.  Redundancies can usually be identified early on, allowing for normalization exercises to take place where the best, combined or new component survives.  It’s also necessary to accept a hybrid approach to the resulting components.  For example, a component can merely be “wrapped” with an interface, thus exposing it’s functionality in an agile way however, the implementation remains non-partitioned and largely unaffected underneath.  Interfaces can also be partly-partitioned, meaning there is some implementation that has either been refactored or added to allow greater granularity to the exposed functionality.  Interfaces can also be “composite”, meaning they are a sum of multiple components.

The data modeling is no different.  Lots of iterations resulting in further normalization each time a new decomposed component list is published.  Data can be encapsulated, just like components can.  If a particular complexity is irrelevant to the whole, those details can be hidden for simplification.  The data model can group elements into an actual data-service, meaning common access to information (just data, no functionality), or groupings can be context for the various components resulting from business process decomposition (which is where functionality is exposed).

Throughout the process, various issues typically arise.  Education in advance usually helps mitigate those issues before they bubble into tangible problems.  Normalizing the redundant parts uncovered, governance for policy management, version control, quality/integrity management, proper expectations are some of the education needed in the beginning to ensure a successful decomposition effort.  The right tools are also a necessity to holding the effort together, (i.e. having established standards, a center for testing compliance to those standards, a registry to keep track of all the parts coming out of a decomposition effort, business process management to ensure composite solutions can be orchestrated and deployed from all the parts and agility created, etc…).  SunGard offers it’s process, tool sets and overall infrastructure for SOA in an offering called “Infinity”, which contains all our experiences, maturity and lessons-learned from our on-going and “frightening” job of making software in the Financial Services industry flexible, agile and “infinitely” configurable.

Posted in Software, Trends | No Comments »

Whatever-as-a-Service

Wednesday, April 11th, 2007

Since my posting last month regarding Software-as-a-Solution I’ve noticed a number of additional variations on SaaS, and I find the many options being discovered and used in the real-world very interesting.  In this article I cover various other forms of XaaS (X-as-a-Service) or WaaS (Whatever-as-a-Service).

 

Various forms of service offerings have appeared on the Enterprise scene, solutions, information, applications, hybrids, infrastructure, etc.  The continual evolution of SOA related to services continues to expose new and interesting domains.  If you define Software-as-a-Service as merely a hosted web-application delivered in a subscription model (pay-as-you-go), you’re missing an entire movement that is appearing to grow ever more prevalent in many Enterprise-SOAs.

 

Information-as-a-Service, for example is access to business entity information (at real-time).  In Financial Services this may be customer portfolio information, market data, security master, etc.  The information sources are diverse (multiple disparate locations) but abstracted so the information service provides an abstraction to the original location and format, allowing the information to be accessed in a SOA way.  Formats may include XML, SQL, files or other domain specific formats such as FIX, SWIFT, etc.  Web-oriented information used in mashups and other compositions may be WSDL via SOAP over HTTP, or REST, RSS, etc.

 

Infrastructure-as-a-Service is yet another twist, which focuses on low-level capabilities (plumbing, or the stuff required to be there for any application to run, yet non-specific to applications).  These services include identity services, logging services, reporting services, etc.  This form of service shouldn’t be confused with the comprehensive footprint of the information fabric (the virtualized data layer).  I’m not sure the data layer can be a service (it’s in the information management domain).

 

Business services, application services, etc… (the list goes on) are yet other types of services showing up in the SOA scene.  Business services deliver a business capability, like a web service that submits a trade to an ORM, or a service that invokes a rules engine to check the suitability of a trade, these would be business services.  Application services would be those necessary components required for an application to function without relevancy to the business domain, but specific to the application.

 

These services even get sub-classed further, for example Information-as-a-Service may have to assume a common data model for highly shared data, as opposed to passing around huge amounts of data (imagine a data feed containing real-time exchange data replicated to every service requiring a price, as opposed to a shared master table).  Said common data may need to be the ‘context’ for a service to execute, and may contain results; therefore the model is an attribute or characteristic of the service, perhaps sub-classing the service.

 

My whole point is that there are all kinds of services.  I’m amazed at how simplistic most SOA approaches try to normalize services into the same thing, (like how many Registries treat every service as basically a simple invocation of something, where a little data is provided and an output is expected).  In order for services to be useful in real-world business solutions, the realization of the reality of the various kinds of services is essential.

Posted in Software, Trends | No Comments »

Blog RSS address moved

Wednesday, April 11th, 2007

Just fyi, if you’re subscribed to my blog RSS feed, the address is now:

 

http://www4.sungard.com/blogs/darrenWesemann/feed/

 

Posted in General, Software, Misc | No Comments »

Software-as-a-Solution

Tuesday, March 6th, 2007

A popular theme these days is the software distribution method known as Software-as-a-Service.  Although it’s clear the capabilities of SaaS are finding tremendous uptake with applications such as CRM and related offerings, there are markets that require an enhanced context for software services to continue to grow.  Financial Services, for instance require highly configurable services due to the large amount of custom integration involved in software deployments.  This yields an emphasis on solutions that meet specific business needs, as opposed to the services that make up the solutions.  Perhaps Financial Services is not completely unique in this regard; however the need for a solutions-focus is emphasized due to complex integration requirements.  In this blog I’ll describe what SunGard has planned for delivering software solutions comprised of highly configurable services.

I’ve commented several times on SaaS (see SaaS Infinitely Configurable, Trend of Software Delivery Methods, Software-as-a-Service vs ASP).  Today, I’d like to extend those comments into the context of configurable solutions.  This is an area near and dear to my heart, because in the financial services sector, (the market I’m exclusively focused in), the SaaS model as it is generally offered, mostly doesn’t fit.  The reason is a CRM-like application doesn’t model the solutions servicing the needs of the typical financial institution.  In the financial services space, seldom can you simply log-in to an online application without intense data and functional integration to existing back-office systems.

Accordingly, software services offered in a hosted or SaaS fashion must not be the endgame, but rather components of what should be emphasized instead, which is the Solution!  So, from our perspective, it ought to be Software-as-a-Solution.

Back in November, SunGard held an event with industry analysts.  At the event, we announced our plan to address a solutions-focused Software-as-a-Service offering for our SOA enabled components.  You can read what was presented here.  We call the offering Infinity.  I’m very enthused about this direction, as it provides configurable components with the endgame being the solution, which directly solves a customer’s business problem.  This is what is needed to make SaaS work with financial institutions.

Posted in Software, Trends | 1 Comment »

« Previous Entries
Next Entries »