Microservices Archietcture(MSA) : Design & Development Approaches

In the recent past there has been flood of requests from major retailers who are using the premium ecommerce packages like Oracle ATG, IBM WCS, SAP Hybris & Magento etc to decouple the functionalities from the monolith deployment by moving the architecture approach to microservices. Decision for separation has to be made with lot of care & consideration as there will be too many factors at play like hosting(traditional + cloud), scaling, data modeling, replication to be used, session, cache management etc. Apart from the above listed factors the custom implementation of the solution would also come into play as we design the solution approach for incremental microservices transformation journey for the enterprise.

As we all are going through the learning curve of developing and deploying the distributed applications this journey provides lot of insights into design & development approaches to the various scenarios & problems. I have tried to summarize some key notes which can be useful during MSA implementations. Please share your learnings and experiences to make it more comprehensive.

Below are the multiple focus areas during designing the incremental microservices journey and list can be improved further based upon the experience we gain from the real ground:

  • Domain Architecture
  • CQRS (Command Query Request State) : Separates reads & writes to the data
  • Event Sourcing : Event based architecture
  • Hexagonal Architecture : Logic Kernel & Adaptor based
  • Resilience & Stability
  • Technical Architecture

While focusing on the above areas some things that will came into light during team discussions are listed below…

  • How to manage the service versions?
  • Designing for performance
  • Designing for integrity
  • Designing for failures
  • Service Boundaries
  • Designing for adapting to third party integrations/Products
  • Operational Effectiveness
  • Health checks
  • Performance Metrics & Alerts
  • Resources utilization & effectiveness
  • Logging & Error, Exception Handling
  • Event & Transaction Tracking
  • CICD for packaging & deployment
  • Hybrid Infra model support ( cloud + physical )

Domain Architecture

  • Defines how microservices will implement core domain based functionalities
  • No rigid approach (allows teams to decide the design & structure enabling them to work independently)
  • Easy to understand, Simple to maintain & replace
  • Defines how the entire domain is split into various areas (leading to mapping one team for the changes)
  • Efficient domain architecture will require low co-ordination and communication among teams

Cohesion

  • Domain architecture of overall system influences the domain architecture of each microservice
  • Microservices should be loosely couple with each other and have high internal cohesion
  • With respect to domain it should follow SRP (Single Responsibility Principle)
  • Loose coupling & high cohesion

Encapsulation

  • Internal details are hidden
  • Accessed via defined interfaces
  • Helps in easy modification
  • In microservices context one microservice will never allow other microservice to access internal data
  • Each microservice understand the interface to other microservice

Domain Driven Design

  • Functionally structures the overall design of microservices

Transactions

  • Bundle of actions (Execute Everything or Nothing)
  • One transaction one microservice
  • Transaction across microservices (by leveraging messaging based arch or design)
  • Very hard to achieve the one transaction one microservice level with domain design

CQRS (Command Query Responsibility Segregation)

CQRS is an architectural pattern for developing software where the system is segregated for the steps or operations dealing with reads & writes (very different from old monolith CRUD architecture where single component has the ownership for reads & writes)

  • Each system has a state which can be saved which requires reading(queries) & writing(command) the data
  • Command & Query can’t be synchronous in nature resulting in CQS (Command Query Separation)
  • Cache’s support the query operations

CQRS

Principles of CQRS Arch Benefits of CQRS Arch Challenges of CQRS Arch
Event Sourcing

Event Driven Architecture

Loose Coupling

High Cohesion

Domain Driven Design

SoC (Separation of Concern)

Domain/Busines Focus

Scalability

Simplification

Flexibility

Command & Query can be use different technologies

Read & Write separation

Data consistency

Expensive ( Development & Infrastructure cost will go up)

Transactions with read & write steps are difficult to implement

msafig12

Microservices with CQRS

  • The communication infrastructure can implement the Command Queue when a messaging solution is used. In case of approaches like REST a Microservice has to forward the commands to all interested Command Handlers and implement the Command Queue that way.
  • Each Command Handler can be a separate Microservice. It can handle the commands with its own logic. Thereby logic can very easily be distributed to multiple Microservices.
  • Likewise, a Query Handler can be a separate Microservice. The changes to the data which the Query Handler uses can be introduced by a Command Handler in the same Microservice. However, the Command Handler can also be a separate Microservice. In that case the Query Handler has to offer a suitable interface for accessing the database so that the Command Handler can change the data.

When to choose CQRS ?

  • Event based integration is develop & maintain in the long run
  • If there is heavy imbalance between number of reads & writes in the given system
  • While building multi-view based systems (Web, Mobile, API etc )
  • Complex CRUD operations in existing system
  • If business process has lot of events and records would be required for further analysis
  • Can live with Eventual consistency

Event Sourcing

Event Sourcing is an similar architectural pattern like CQRS, state is recorded as the sequence of events in the system resulting from actions/steps of domain. Events help in rebuilding the state of the actors with in the domain at any point of time.

CRUD stores the current state(not events) of the entity not the sequence or history of the events

  • Events are messages
  • Can be modelled in separate classes
  • Event history is maintained (persists events)
  • Event stores provide read and write capabilities
  • Used for interaction between components
  • Events are immutable ( no modification or deletion)

 

Principles of Event Sourcing

Benefits of Event Sourcing

Challenges of Event Sourcing

Persisting state of the events

Event Driven Architecture

Loose Coupling

High Cohesion

Domain Driven Design

SoC (Separation of Concern)

Simplicity

Easy Integration

Event state traceability(audit trail)

Performance

Flexibility

Consistency (Business drives the level of consistency not the technology)

Parallel updates

Validation

Event Sourcing
  • Event Queue : Sends all the events to recipients (mostly messaging middleware)
  • Event Store : Saves all events using them it can reconstruct state
  • Event Handler: Business logic which responds to the particular event(s)

Hexagonal Architecture

 “Focuses on the core logic of the application i.e. business functionalities with well-defined interfaces which are available for the users & administrators”

  • Alternative to the layered architecture which has UI, Business Logic, Persistence
  • Has adaptors which interact with logic via ports
  • Inbound & Outbound communication
  • Some of the adaptors available for users, data, events & administrators are….
    • UI Adaptor
    • REST Adaptor
    • Test Adaptor (Enables isolated testing harness)
    • Database Adaptor
    • Custom Adaptor (Resilience & Stability)
Adapter (From GoF it is also called as Wrapper)
msafig14

Resilience & Stability

  • Due to the distributed nature of the microservices architecture the system always can run into the risk of cascading failure resulting in hostage situation for the applications accessing the services
  • Distributed architecture relies on the network & cloud servers which can be highly unreliable considering the risk of failure system should be designed in such a way that probability of failure can be masked i.e. building the resilience into the system/application to ensure the availability too.
  • Patterns that can ensure the availability of the services are:
  • Bulkhead
    • Bulkhead has been used for several centuries in ship building area, it creates the watertight compartments that can hold water in case of leakage to avoid ship sink
    • Comparison to software architecture the concept allows to build sub-systems which are well shielded from cascading problems if any like overloads, response failures etc
  • Test Harness
    • It is an approach to induce the situation to study the behavior of the application in certain contexts (TCP/IP, HTTP Headers etc i.e some OS or network level issues) most of time these are not considered but addressing these situations can help the overall architecture
  • Steady State
    • Simple and manageable at any point of time, if the systems are not designed for consistent state they might run into problems at some point of time like heavy database storage operations, log & caching operations which have capability to bring down the system by creating storage & operational challenges. Hence it is recommended to have automated control over storage, caching & log operations
  • Uncoupling Via Messaging
    • Asynchronous process based communication helps (without any waiting), approach should be to proceed with other activities without waiting on the response
  • Timeout
    • An individual thread is started which can be terminated after defined timeout
    • Easy to implement compared to other patterns
    • Can be plugged into monitoring systems as well
  • Stability based patterns
    • Use Circuit Breaker, Timeouts & Bulkhead patterns to safeguard from the failures and tight coupling
    • Apply fail fast mechanism to all the microservices to ease the communication wait times
    • Steady State, Fail Fast, Test Harness, Handshaking etc have to be implemented by each microservice
    • Decoupling via shared communication is the way to design
  • Handshaking
    • It is a protocol based feature once leveraged will help us to track the overloaded system & prevent cascading failures, socket based protocols implement these very well but HTTP doesn’t support this mechanism hence it is the responsibility of the application getting built by introducing some type of health check or signaling
  • Circuit Breaker
    • Are also not a complex to implement in the microservices code but a special attention is required during design to make them work high load scenarios & meet the operational needs to monitoring etc..
  • Fail Fast
    • This approach allows the consumer system to timeout on the response and detect the failed state as quickly as possible  with defined error state. Hence validating the state of the incoming request allows to validate and avoid the processing further by confirming the error state, similar to the timeout model
  • Resilience in Reactive Context
    • Reactive Manifesto considers “resilience” as one of the core property of the reactive application
    • Asynchronous processing with error handling & monitoring
  • Hystrix
    • Very good support for resilience
    • It is a java library given by Netflix shared under Apache license
    • Internally it uses Reactive extensions for Java (RxJava)
    • Implements timeout & Circuit Breaker
    • Hystrix dashboard shares info about state of the thread pools & circuit breaker
    • Can be embedded in commands implementation
    • Library provides rich annotations
    • Leverages thread pools (for each microservice) – ensuring the support for bulkhead
    • Non java implementations can use Sidecar Hyrstrix
    • Can configured for monitoring single hosts as well as cluster mode

Technical Architecture

Technical architecture of the microservices can be designed independently, tech stack or frameworks need not be common i.e. truly ployglot support. Operational or non-functional needs can be collaboratively designed to support & maintain the system.
  • Process Engines
    • SOA based orchestrated process engines can be leveraged for microservices as well
    • Microservices are designed for one domain i.e. one Bounded Context
    • Microservices should not be designed just for integration or orchestration else it will violate the SRP and the changes for microservices will be become complex to do and maintain over the period
    • In case of collective business context multiple microservices should be implemented where each microservice supports one business functionality and used in conjunction to support multiple functionalities
    • Integration only microservices should be strictly avoided
  • Statelessness
    • In a distributed architecture stateless microservices are very useful
    • Microservices should not save any state in their business logic (State can be maintained in database or client side)
    • Having the stored state helps to recover from failures by replicating/recreating the state from database
    • Helps in creating multiple instances
    • Easy replication
    • Load distribution to the instances
    • Versioning : Old version can be shutdown or replaced without migrating state
  • Reactive
    • Reactive properties fits very well into the microservices approaches/needs
    • An application/systems which is reactive should have some defined properties like:
      • Resilience : Ensure the availability & failure tolerance
      • Elastic : Dynamically scalable at runtime
      • Message Driven : Message based communication
      • Responsive : Fast response times with fail fast setup
    • Actors/Processes can send messages to each other

      • Non-blocking I/O communication(Asynchronous)
    • Reactive Technologies

      • Vert.x ( Java JVM based but also supports various languages like Ruby, JavaScript, Groovy, Scala, Clojure & Python)
      • Reactive Extensions : RxJava & RxJS
      • Scala ( Reactive framework Akka & Web framework Play)

msafig15

 

Reactive is just an alternative approach available of the microservices but it is not the only option. Microservices can also be implemented without “reactive” approaches with traditional programming models, resilience can be achieved by using various libraries & elasticity is implemented using VM’s or Containers. Messaging is already used in traditional models. Reactive systems have real advantage in the case of “responsive” implementation

References:

 

Advertisements

Microservices Architecture(MSA) : Security, Authentication & Authorization

Problem : How to handle security with authentication & authorization in MSA?

MSA requires uniform security to ensure microservices can communicate and collaborate with-in and outside the system. Each microservice needs info about who triggered the call for system usage. Authentication & Authorization are two core components of the security apparatus both are independent of each other.

Authentication : Identity of the user

Authorization : Decision on user is allowed to access & execute actions

  • Individual microservices should not perform authentication(central server approach can be adopted)
  • Authorization enables the interplay between microservices hence it would require the user groups, roles etc to created & administered collectively
  • OAuth2 is a widely supported protocol( Google, Microsoft, Twitter, XING, Yahoo etc)

OAuth2 solves the authentication & authorization primarily for human users for enabling it for microservices based system would need additional steps like:

  • Microservice to microservice communication protected by SSL/TLS against wiretapping (by encription) – REST supports such protocols
  • Certificates can be used to authenticate clients(non-humans)
    • Certificate Authority (CA) creates certificates & can be leveraged to verify digital signatures (SSL & TLS support certificates)
  • API keys (similar to certificates, keys are shared with external clients to allow them access to the system) – OAuth2 supports this through Client Credential
  • Firewalls : Use for controlling the microservice to microservice communication, helps in intrusion scenarios to enforce restrictions
  • Intrusion detection mechanisms (part of the monitoring system setup)
  • Datensparsamkeit : Concept enforces the restriction on data collection fields (collect the data which is absolutely required rest all are not store in the system)
  • Hashicorp Vault : Tool
    • Secret password
    • API keys
    • Keys for encryptions or certificates
    • Integrates with Hardware Security Modules (HSM’s),SQL databases, AWS IAM, can generate keys for AWS cloud by itself

Additional security scenarios for consideration:

Each micoservice has to ensure the below things in its design & implementation:

  • Integrity : Data integrity is the responsibility of every microservice it can be achieved by signing the data
  • Confidentiality : Ensures data modifications are not denied (again by using the user keys)
  • Data Security :
    • No data loss
    • Backup & disaster recovery solutions
    • Highly available storage solutions
  • Availability : Resilience approach should address the failure scenarios to ensure high availability

Most of the times above scenarios are ignored or missed out while designing security approach for the systems.

Questions to probe further during design discussions:

  • Session management implementation
  • What is the microservices communication & service discovery model?
  • User experience can change or should remain same?
  • Are you breaking the monolith to SPA(Single Page Applications)?
  • Do you have mobile clients?
  • How is identity and permission info conveyed to a service?
  • How is info decoded and interpreted?
  • What data would be required to make access decisions(ACL’s, roles, user accounts etc)
  • How is data managed?
  • Who is responsible for storing and retrieving it?
  • How to verify the tampering case?
  • Who all will interact with the services?
    • Users, Devices, Services, Applications, External Services,

Approaches available:

Evaluate all the approaches to see the proper fitment, multiple factors are at play, traditionally authentication in monnolith(heavyweight architecture) world has been always sateful service but MSA is lightweight distributed architecture but works mostly on stateless services, below are the approaches to work with stateful microservices:

  • Old world options available are….
    • Basic HTTP authentication (has drawbacks )
      • $ curl “https: //$username:$password@host/resource
    • Certificate based security : Supported in Sprint Security (X.509) – (has drawbacks )
    • Network security – (has drawbacks )
    • Custom Authentication Token
  • In MSA’s mostly deal with stateless stuff, we need to focus on what is stateful and stateless & how do we achieve statelessness?
    • Authorization servers are often stateful (they store issued access token in db for future checks)
    • By issuing JWT tokens as access tokens will help us to get statelessness to the interactions
      • JSON (JWT) is a secure way to encapsulate arbitrary data that can be sent over unsecure URL’s
  • Leverage JWT (JSON Web Tokens)
    • URL-safe
    • Exchanged between two parties
    • Claims are encoded as JSON object which is digitally signed by using secret between two parties
  • Use some Redis/Memcached db for user sessions in all the frontend microservices(this will ensure user authenticated once will have continuity for the following sessions)
  • Tokens (Expiry & Non-Expiry) : For expiry always pair with refresh tokens
  • API firewalls/reverse proxies are used for token translations
  • OAuth 2.0 is an open source delegation protocol allow secure authorization in simple & standard from desktop, web, mobile applications
    • In MSA context:
      • Authentication can be a microservice itself
      • Authorization is a common functionality across MS’s
    • Implicit grant (Frontend application)
      • For web applications running on the browsers with AngularJS or mobile apps
      • Gets access tokens directly
      • Similar to code grant but client gets access token directly
    • Client credentials grant :
      • Service-to-Service
      • Application authenticates itself using clientid and client secret
    • Password credentials grant:
      • Client collects username & password to get an access token directly
      • Only viable for trusted clients
    • Authorization code grant:
      • Server based application
      • Redirect flow for web server apps( to authorization server for code)
  • Frontend applications
    • Use implicit grant
    • Use HTML 5 localstorage for access and refresh tokens

oauthfig1

Solution Frameworks:

  • Netflix Zuul
  • OAuth : It is not for authentication & authorization but for scalable delegation protocol where actors involved are (map the numbers beside approach to the figure shown above ):
    • Resource Owner(RO)
    • Authorization Server(AS) – (2) (5)
    • Resource Server(RS) – (4)
    • Client – (1)(3)(6)
    • Access tokens are JWT (JSON tokens help in achieving the statelessness)
      • By Value(Inside the network)
      • By Reference(Outside the network)
  • OpenID Connect
    • Resource Owner(RO)
    • Authoriztaion Server(AS) – (2)
    • Resource Server(RS) –  (5)
    • Client – (1)(3)
    • MyMail.com – (4)(6)
    • Sessions – (5)
  • JSON Web Tokens
  • Sprint Security REST plugin
    • Spring Security core support – has inbuilt support for Oauth2 standard  that can be extended or customized for our implementation needs
    • Login & Logout REST endpoints
    • Token validation filter
    • Memcahed token storage
    • Partial implicit grant support for 3rd party providers
    • Oauth 2 support
  • SAML assertions (similar features to OAuth2)
    • XML focused
    • Standard based like OAuth
    • Supported by Spring Security
    • Standardized request signing
    • Enterprise usage
    • Drawbacks
      • Cumbersome setup for servers & clients
      • Headers will have large amount of XML data
      • Complexity for developers
  • Datensparsamkeit : Approach
  • Hashicorp Vault : Tool

(Image Source : OAuth.net )

oauthfig2

Solution Design:

There is no one size fit all the solution for this scenario, we have to adjust our solution according to the context & need of technical setup on the ground

  • Core problems in MSA :
    • Identifying user
    • Creating sessions
  • Leave authentication to Oauth or OpenID server
  • Make all MS’s accept JWT’s (with translate using reverse proxy for by reference to by value translations)
  • Ensure that….
    • Everything is self-contained
    • Standards based
    • Non-reputable(don’t trust anything)
    • Scalable
  • JSON (JWT) can be used for…
    • Single Sign-On
    • Secure payload transportation (obscure url parameters or POST body elements)
  • JWT for achieving statelessness
    • Instead of storing access token in a stateful way, do it with JWT
    • On the client browser securely store the JWT-encoded access tokens ( this is helps us to achieve one of the basic rule of REST i.e. state transfer
    • JWT claims can be signed
    • JWT can leverage JWE (JSON Web Encryption)

 Summary:

Lightweight distributed architecture based services demand lightweight infrastructure, security is very important but should be unobtrusive, spring security makes it easier for the developers to understand the low level needs of security communication & implementation. Oauth 2.0 being a standard helps to have common streamlined approach to solution implementation.

  • Spring Security Oauth
  • Spring Security SAML Assertions
  • Sprint Session
  • Oauth2 seems to be evolving as the alternative it is
    • Extremely simple for client (lightweight)
    • Access token have information other than identity only
    • Resources are free to interpret token content
    • Standards based hence changes be decoupled for uniform implementation
    • Supports bearer tokens
      • Centralizes account mgmt. & permissions
    • Supports Authorization Server, Resource Server & Client Application
      • Examples
        • Facebook – Graph API
        • Google – Google API
        • Cloud Foundry – Cloud Controller

References:

Microservices Architecture : Integrations & Communications

Systems can’t be built in isolation, each part of the system needs to communicate with the components and also integrate with other parties to provide & seek the services. Microservices are no different they need to be integrated & have to communicate with other services & systems, this happens at various levels as shown below…let us explore each of these to understand the details of implementations & challenges, it is going to be long post as we need to see the various options available for the challenges to be addressed during integration scenarios. I would advice a cup of coffee won’t be a bad idea while reading  🙂

msafig3

User Interface (UI) Integration

  •  Each microservice can have its own UI
  • Single Page Applications(SPA)
  • Single HTML Page
  • JavaScript (udpates page dynamically)
  • Logic in JavaScript
  • Useful for complex interactions & offline ability requirement

Single Page Application(SPA) Technologies

  • AngularJS (Runs in browser, given by Google under MIT open source license)
    • Provides bi-direction UI data-binding which tracks changes at
      • View Level (Renders in browser)
      • Attribute Level (Code Level)
  • Ember.js (Similar to AngularJS with it s own module Ember data it provides MDA(Model Driven Approach)  for accessing REST resources. Also available under MIT open source license)
  • Ext JS provides MVC features & UI compose like Rich Client Applications. GPL license v3.0 ( Free for non-commercial implementation, for any commercial implementation license needs to be obtained from Sencha)

Microservices & SPA’s

  • SPA per Microservice
    • Each microservice can have its very own SPA which calls microservice via JSON/REST interfaces
      • Tight coupling of SPA’s is difficult
      • Switching between SPA’s needs browser app reload (not a recommended integration)
    • SPA’s should have uniform Authentication & Authorization to avoid the need for multiple logins , OAuth2 & Shared Signed Cookie can help solve these kind of problems

msafig4

Uniformity with Asset Server

  • Each SPA is assembled with its own UI
  • SPA’s of microservices can use the resources(JS,CSS etc) from asset server to enable uniform UI
  • Proxy Server helps in masking the security rules of different URL’s (helps in giving uniform experience)
  • Separation of assets will result in new problem of dependency(All the changes to the assets will require changes in microservices as well)
    • Risk to independent deployments
    • Scenario best avoided
    • Reduces dependencies in frontend

“Asset Sever is more a problem than a solution”

msafig5

JavaScript in SPA can access the data

  • From the same domain as the code originates(called as origin policy)
  • JavaScript can’t read data from multiple domains
  • All microservices are accessible to the outside under the same domain due to proxy(this is no limitation)
    • Policy has to be deactivated when UI of a microservice is supposed to access the data of different microservices
      • Solution 1: CORS (Cross Origin Resource Sharing) solves the origin policy problem
      • Solution 2: Have all SPA & REST services under one domain (here asset server is the best option to enable the solution)

Single SPA with multiple microservices

msafig6

Binding of UI would be required as the functionality of the pages might come from different microservices which are composed of the module(JavaScript, CSS) of specific microservice that has all the UI formation (assuming that implementation is AngularJS based)

  • AngularJS supports multiple modules for different SPA units/parts hence facilitating the each MS to have its own UI control module

Drawbacks:

  • Dependency across SPA & microservices for build & deployments( requires lot of coordination) – Better to avoid this setup
  • UI based coupling of microservices results in shared deployment artifact – Better to avoid this setup

“One advantage of this approach is SPA brings together UI from different microservices & shows to the users without any intervention “

HTML Applications

Each microservice has one or more web pages(HTML, JavaScript, CSS etc) generated on the server (difference from SPA is that only a new HTML web page is loaded not an application/module by the server)

ROCA (Resource Oriented Client Architecture)

msafig7

  • Handles JavaScript & dynamic elements in HTML UI
  • Alternative for SPA’s
  • JavaScript plays limited role of optimizing the usability of pages(facilitates & add effects to the web pages)
  • Core assumption application should be functional without JavaScript
  • ROCA reduces dependencies (core philosophy that web should work on HTML over HTTP)
  • In various microservices coupling can be enabled by links (natural integrations unlike SPA’s)

Routing based integration

  • From outside the mircoservices should appear like a single web application(with one URL)
  • Also enables the assets to be shared without violating same origin policy of various assets in the system

msafig8

Function of the router is to redirect the requests from external/internal parties to appropriate microservices

  • We can leverage Zuul for achieving this functionality, also reverse proxies can also help(like webservers nginx, apache httpd that direct request to other servers)
  • Request can be modified & URL’s are altered which is very easy in Zuul to extend & implement

NOTE: When the rules in router become complex it can kill the purpose of isolated deployment strategy hampering independent development & deployment.

JavaScript for HTML Layouts

  • AJAX can be leveraged to load the content of a link from another microservice(s)
  • Link based approach can be used
  • Facebook BigPipe :
    • Optimizes load time
    • Composes web pages from pagelets
    • JavaScript is used to replace elements of webpage(DIV elements etc)

Frontend Server based integration of Microservices

msafig9

  • Frontend Server assembles the HTML webpage from HTML code snippets generated by microservices
  • Assets like JavaScript, CSS etc can be stored in frontend server
  • Edge Side Integrations(ESI) concept helps by providing simple language to collate HTML from various sources
  • ESI cache holds static & dynamic content of a page (this mix of static & dynamic contents is key for assembling pages)
    • Proxies & cache (Varnish & Squid implement ESI)
  • Server Side Integrations (SSI) can be another alternative, but SSI doesn’t have cache support) and only available in webservers
  • Each microservice can deliver HTML snippets which can be integrated on the server
    • Apache httpd supports SSI’s with mod_include
    • Nginx supports SSI’s with ngx_http_ssi_module
  • Portlets can be used as an alternative solution for the content assembly ( JSR 168, 286) in this case but it is very complex and the portlets are implemented differently in various servers (For microservices this solution is best not to be considered) Also most of the portlet implementations restrict us to java technology only…

Integration Scenario with Mobile & Rich clients

Mobile & Rich client applications present a different challenge in the current context microservices integration as they have separate client side installer which is required to access the services whereas web implementation is more universal and generic with central management of the UI assembly with HTML snippets shared by microservices. Also the entire mappings can be controlled by single coordinated client & server deployment.

  • Mobile, Rich or desktop clients requires separate installation for the client (which is predominantly monolith in nature) which has to offer/manage the interfacing with microservices
  • These client requirements might drive/dictate changes in the microservices development & deployments hence it is advised to bring some modularity into the context
  • We can’t have multiple client app deployments for the feature changes in microservices as the users will have direct impact or overhead requiring updates on the devices (this isolation development approach is considered in SPA’s for client & server development but the same is not recommended)
  • Both client & microservices can’t be isolated coordination is required to rollout and address the channel based needs at optimum levels

Organization Scenario

  • Mobile client development teams are different from the microservices teams (adding the client specific developers in each microservice team is not a great approach)
  • Mobile platform team needs to interact with all the microservices implementation teams which requires lot of communication which is not recommended by microservices approach

msafig10

One probable solution can be microservices team develops the features for web and the same will be available to mobile app when it is released but the catch here is that each microservice needs to support different features for web & mobile (to avoid the scenario we can have similar features across channels but in the domain context it might not be the case as the requirements/usecases might vary)

 

 

Agile Ecommerce Initiative: Hybris Multichannel Accelerator

Back after long blog fasting 🙂 was busy farming on some books & technologies. One of the ecommerce package tools, which caught my attention while working on some solutions for the customers is Hybris. It announced loud entry into the ecommerce world and the existing leaders are already feeling the heat. Being a multi-channel platform it has also provided the effective user tools to quickly setup the sites. I being a technology illiterate was able to configure and customize the multiple sites just in one day that was too fast, it was possible only due to HMCA(Hybris Multichannel Accelerator).

We have to really appreciate this agile initiative from Hybris that is helping both customers & partners to quickly pilot the POC to see the business process functional. As part of my learning series I am putting together my steps that might help someone who is looking for some guidance…let me know how it worked for you. Generally I’m Linux fan but due to server availability restriction I had to try it on Windows 7 (tried on both 32 & 64 bit versions), below are the steps. I predominantly experimented with B2C full workflow, will dive into B2B too.

1) Sign-up for the id on hybris wiki page(Takes 2-3 days for partner approval from Hybris) [https://wiki.hybris.com]
2) Login with credentials on to [https://wiki.hybris.com]
3) Download hybris-multichannel-accelerator-standalone-4.7.0.zip file from https://wiki.hybris.com/display/downloads/Download#Download-GeneralPackages
(hybris Multichannel Accelerator Suite 4.7.0 – B2C)
4) Extract zip file to any your desired directory ( For example D:\HMCA)
5) CD D:\HMCA\Hybris\bin\platform
6) Windows look for setantenv.bat & run it
a. This will set ant home on your system
(Setting ant home to: D:\HMCA\hybris\bin\platform\apache-ant-1.8.2
Apache Ant(TM) version 1.8.2 compiled on April 21 2011)
7) Run ant clean all, input data to move ahead
[Input] Press [Enter] to use the default value ([develop], production)
8) Press enter for [develop]
9) Time to test our installation 🙂
D:\HMCA\hybris\bin\platform>hybrisserver.bat
10) http://localhost:9001/
11) Initialize your setting using the “Initialize” button on the page
Note: Leave setting to default as suggested by system, just to feel good about first installation and configuration [Experiments later ;-)]

Hybris Multichannel Accelerator functionalities

  • B2C,B2B Workflows,Storefronts,Cockpits,Search & Navigation,Promotions Setup Interfaces,Cross Sell,Checkout,Payments Gateway Integrations,Reviews Integrations,Order Management,Fraud Engine, Product & Catalog Management,Reporting, Store Locator Integrations,Social Platform Integrations etc….

    After basic installation & configuration below are the screens for management console & admin console post init phase with electronic & apparel sites.

Things I liked most are….Let me know your good & bad experiences with Hybris.

  • Strong flexible architecture for customizations
  • Support tools for development & management
  • Great Unit & Integration Test support
  • Consistency across channels
  • Simple & Clear documentation
  • Supports both B2B & B2C flows effectively


This slideshow requires JavaScript.

How to leverage pinterest as a social commerce accelerator?

Pinterest is new site that gives registered customers (purely by invite, they are rapidly growing I am watching their strategy when they will make it open or continue it same wayJ) to create and customize pictorial boards. It is quick and easy way to share & use the existing content of your products. Allows you to categorize products and acts as bulletin board marketing tool for your products.

Pinterest

If you can visit and see there are already few million users having private & public boards. Pinterest is providing the retailers to share their products and engage customers on social platform.

Now let us talk some business J for ecommerce sites, What is in Pinterest for a online retailer?

how to leverage it, here are the few ways:

  • Create your own pinterest brand board : Helps you to share the product information with customers. It is catching up viral already on the fashion products. I would say a new way to promote your products J
  • Add “Pin It” button to Product Detail Pages of ecommerce sites: Enables the visiting customer to quickly share the product images with PDP links. With this sharing done by customer whenever someone else in pinterest community clicks on your product images they get redirected to your site pdp hence increase your traffic, promoting your products in turn can result in revenue growth.
  • Announce your pinterest to customers: Use social media options like twitter & facebook. Add “pinit” buttons to your promotional contents.
  • Organize contests, brand buzz events on pinterest : Allows you to engage customers to create a brand buzz online.

Pinterest is already creating waves in the online promotions of products, generating traffic via bulletin board marketing approach, very soon there will be lot ecommerce sites using this on their sites.

This is a new trend developing keep a watch on the best ways to cash it for our customers. Please post your experiences if any on the same lines.

Spring & Hadoop becoming de-facto for enterprise apps

Spring Hadoop an integration of the Spring Framework and the Apache Hadoop platform designed to give developers an option to build distributed processing solutions with Apache Hadoop. Wth lot of focus around cloud computing to leverage the real value of the infrastructure available at disposal has resulted in lot attention towards high performance computing.

Hadoop

Apache has provided the open stack format of Map Reduce Framewrok known as Hadoop. It provides:

  • Hadoop Common: The common utilities that support the other Hadoop subprojects.
  • HDFS: A distributed file system that provides high throughput access to application data.
  • MapReduce: A software framework for distributed processing of large data sets on compute clusters.
  • Avro: A data serialization system.
  • Chukwa: A data collection system for managing large distributed systems.
  • HBase: A scalable, distributed database that supports structured data storage for large tables.
  • Hive: A data warehouse infrastructure that provides data summarization and ad hoc querying.
  • Mahout: A Scalable machine learning and data mining library.
  • Pig: A high-level data-flow language and execution framework for parallel computation.
  • ZooKeeper: A high-performance coordination service for distributed applications.

Talking about the “new wave of data-driven applications” industry focus has been more on so-called Big Data issues, there is interest in VMware delivering a streamlined programming model that could make Spring a natural way to integrate Hadoop systems into the enterprise app landscape. “Spring Hadoop brings the benefits of Spring — simplicity, ease-of-use — to Hadoop by providing a comprehensive, lightweight framework that will allow developers to easily build solutions around the Hadoop platform,”.

The situation here is that of huge data volumes further “data access” choices in enterprise applications have grown exponentially (i.e., there is widespread secure access to the corporate data center via smartphones, tablets, laptops, and dedicated mobile devices of all kinds) — and this in many senses sums up the challenge brought about by Big Data as we know it today.

VMware has chosen spring to answer these new data challenges & Spring continues to focus on enabling enterprise Java developers to incorporate new data access patterns into their applications through the Spring Data projects.

Key aspects of Spring Hadoop include:

  • Support for configuration, creation, and execution of MapReduce, Streaming, Hive, Pig, and Cascading jobs via the Spring container
  • Comprehensive HDFS data access support through JVM scripting languages (Groovy, JRuby, Jython, Rhino, etc.)
  • Declarative configuration support for HBase
  • Dedicated Spring Batch support for developing powerful workflow solutions incorporating HDFS operations and all types of Hadoop jobs
  • Declarative and programmatic support for Hadoop Tools, including FsShell and DistCp

VMware is strategically promoting the integration of Spring & Hadoop in its virtualization solutions. We need to wait and watch how this combination will work in the real-time environments, but it is definitely gaining market attention as I have seen some of the ecommerce clients already adopting this solution.

References:

1) Apache Hadoop , Spring Framework & MapReduce

2) http://www.scoop.it/t/hadoop-and-mahout

What is PCI?

What is PCI?

I face this question more often these days, everyone is working on something or other for PCI. To answer the question it is a set of comprehensive 12 requirements (256 sub requirements to be precise) for payment card data security standards (Transaction cards like credit, debit & gifts etc), these are developed by the founding payment brands of the PCI Security Standards Council, including AMEX, Discover, JCB, MasterCard and VISA etc, to help facilitate the broad adoption of consistent data security measures on a global basis.

The PCI DSS is a multifaceted security standard that includes requirements for

  • Security Management
  • Policies
  • Procedures
  • Network Architecture 
  • Software design and other critical protective measures.

This comprehensive standard is intended to help organizations proactively protect customer account data and be compliant with common standards laid down. PCI Security Standards Council keeps enhancing these PCI DSS standards to accommodate the new security learning’s that will help world mitigate risks i.e. get some learning from hackers :-).

The core of the PCI DSS is a group of principles and accompanying requirements, around which the specific elements of the DSS are organized:

Build and Maintain a Secure Network

Requirement 1: Install and maintain a firewall configuration to protect cardholder data.

Requirement 2: Do not use vendor-supplied defaults for system passwords and other security parameters.

Protect Cardholder Data

Requirement 3: Protect stored cardholder data.

Requirement 4: Encrypt transmission of cardholder data across open, public networks.

Maintain a Vulnerability Management Program

Requirement 5: Use and regularly update anti-virus software.

Requirement 6: Develop and maintain secure systems and applications.

Implement Strong Access Control Measures

Requirement 7: Restrict access to cardholder data by business need-to-know.

Requirement 8: Assign a unique ID to each person with computer access.

Requirement 9: Restrict physical access to cardholder data.

Regularly Monitor and Test Networks

Requirement 10: Track and monitor all access to network resources and cardholder data.

Requirement 11: Regularly test security systems and processes.

Maintain an Information Security Policy

Requirement 12: Maintain a policy that addresses information security.

How do we start?

  •  Identify your PCI uses in your business at various level ( application, architecture etc).
  •  PCI Discovery ( GAP assessment) [Includes anything & everything on enterprise network :-)]
  • Baseline [Where do we stand?]

Where are the opportunities for technical teams?

  • Network : Network Segmentation (Enterprise Arch).
  • Data : Database (what data comes in, goes out, how do you store it & access it etc).
  • Applications : Interfacing (Application/Solutions Arch: App/web servers, legacy code etc).

 Who all need to be certified?

Merchants doing credit card transactions need to get PCI compliant but there are levels defined based on the amount of transactions done. For more information you can visit www.pciassessment.org

What does PCI non-compliance mean for business?

  • Heavy fines, legal complications etc.
  • Barred from accepting/processing credit cards.
  • Data Security Compromises resulting in security attacks and
  • Many more …….

Myth about PCI DSS : PCI DSS is a technology or framework that would gaurd us from external threats.

In reality PCI DSS is not a technology or famework instead they are governed standards driven to build the compliance environments across the globe to collaborate in secure ways to handle payment cards. Watch out we still need the robost vulenerability(SCABBA) tests to ensure the code is cleaned before putting it into real use 🙂

Where is PCI now and where is it going?

Current PCI is 2.0, yes it also evolved through versions 🙂 & will keep evolving…

Where it is heading, probably we will see the new PCI standards to certify specific clouds that will deal with PCI & all of us talk to those secure clouds. At this moment things are too cloudy for cloud computing…letz park this for next time 😉 may be those will be cloud certification dayz 🙂

Happy PCI dayz N years to come (It is never last time for PCI, always ongoing….)!!!