Big Data and the Enterprise-A new normal for enterprise architecture
The rise of big data systems is primarily driven by web based application paradigms for the B2C market. The growth of B2B solutions delivered through web based application models is driving a few shifts in enterprise architecture. It is as much about the convergence of two different approaches, as it is about the conflict in basic conceptual models.
The web architecture model drives popular web based applications in the B2C segment. On the contrary, the model widely used in the B2B segment, is an architecture designed and built for an enterprise bus. There are differences in the design and deployment models between these two distinct business segments. The web model does not have the concept of an enterprise bus, as there is no app-to-app interface. Each web-based solution is consumed by itself either through desktop browsers, through mobile browsers or through applications on a device. In other words, there is no business process implemented across a suite of applications in a B2C context.
The B2B enterprise model is built keeping in mind, business process implementation around a suite of applications residing on an enterprise bus. Hence, there is collaboration with interfaces and exchange of messages on a standard bus. When there is an application built in a web architecture paradigm and deployed in an enterprise context for a B2B process; the point of integration and the interface method become the main points of debate. This is because the applications do not talk to an enterprise bus, but only feed a data stream into a specific application in the enterprise. There are instances, where the integration is more tightly coupled, and there is a need to collaborate with another application in the same business process. In these cases, the integration point becomes a matter of debate, both business and architecture wise.
Big data architecture for B2C applications with the web as the development and deployment platform is built based on a distributed processing model, with centralized view of the control. Big data applications use commercial hardware and primarily open source software with distributed processing. MapReduce is the predominantly used paradigm. There is one master and many processing modes with no hard dependencies on the actual HW systems or processor architecture, from an application design point of view. There is no application blending in this model. The classical three tier model of data access, business logic and presentation layer still rule the conventional web applications. However, the addition of the big data stream into the data layer, challenges the tier model on the data side, when it comes to the staging or landing tier for incoming data. The Utopian scenario for big data architecture is to remove the distinction between landing or staging areas, data store and information delivery.The reality is quite different on the ground, due to the diversity of data streams, integration and compliance needs.
The enterprise model is built in collaboration with many applications, and the bus architecture in this context is mainly message driven. It is typical to model an enterprise business as a set of business processes, which are mapped to corresponding data and process models. They are in turn implemented using a technology stack sourced from different vendors, with a high emphasis on standards of compliance. Enterprise architecture in its principle form covers the scope of business process modelling implemented through IT systems, providing a return on investment, and it is by its very design driven by architecture and standards.
There is a growing demand for web architecture based big data systems to integrate with enterprise processes, for new B2B business solutions. This sparks integration versus isolation debate, in the deployment phase for both data and application assets. The current models are more skewed towards data level integration, in terms of a feed in a specified format into an internal enterprise application. Take the popular example of delivering social data insights into a CRM application using XML or JSON interface. This is a lightly coupled model, where there isn’t a need for this social web application to sit on the enterprise bus for tighter collaboration. As the B2B segment matures, CIO’s will see the need for tighter integration with other applications in the enterprise, by means of a message bus. This need for a tighter and hard coupled system is also from the LOB applications, where the processing is blended at both data and method levels.
At present, this is an evolving topic, to re-imagine enterprise architecture in the context of web platforms and big data systems. In other words, we may be in the phase of ‘consumerisation’ of enterprise architecture. And this is perhaps the new normal now!