Thursday 29 June 2017

MongoDB Jolie Connector Part 3 : ObjectID creation and cross collections handling

In the last year I have been working on the development of  MongoDB connector for Jolie ( First Language for microservices ). In my previous two posts I have been looking into the basic CRUD operations and how to program them with Jolie.
In this post I am going to look at the handling of ObjectID and cross collections relationships. The two topics are closely related as MongoDB used ObjectID to cross reference documents between collections.

Assigning ObjectID to a document and adding ObjectID reference to a document.

ObjectID is a MongoDB specific data type that is not native to Jolie , therefore like in previous cases where there was not direct correspondence between MongoDB data type and Jolie data type a  the following semantic solution was adopted  


The node "@type" signal to the connector that it needs to handle the value of the node "_id" as a ObjectID. The node "_id" it is a specific node name that will trigger the the assignment of a unique ObjectID to the document as shown in the following example 


The response from the insert operation will return an structure identical to that used to insert the ObjectID


Of course one should really consider the opportunity of setting an objectID with a custom value, considering the all the drawbacks connected with handling the uniqueness of the value.
If the "_id" node is not defined the insert operation will return the auto generated value; that can be used to create documents cross referencing.
In a similar manner we can set one of the document fields as ObjectID to reference an other document as shown in the code under .


The resulting document looks like


We can see that the resulting document contains the auto generated "_id" and the reference to another object that is desired result.It is also clear that any time we can update the document adding further documents reference as shown in the following code.



The document now looks like


Cardinality of the relationship

When approaching MongoDB from a SQL prospective the first question came to my mind was "what about schema". In the excellent post "6 rules of thumb for MongoDB schema design" the author goes back to basic identifying three types of possible relationships:

  1. One to Few
  2. One to Many
  3. One to Squillions
The first  is not really strictly pertinent to the topic of this post but it comes without saying that One to Few can be easily model with an embedded document.
The renaming two are more pertinent: and I will try to demonstrate to code these relationships with Jolie
In my example  are present 3 collections that model a small part of a e-commerce solution:
  • CustomerData
  • CartData
  • SearchData
The CustomerData document ( see below ) will contain an array field cartHistory containing the reference to CartData document
   


The document for CartData( see below ) also contains a reference to a forth collection ProductData that will be not specified in this post.


In both cases the documents are modeling a One to Many relationship The SearchData document ( see below ) acts as log of all the search activity logging the visited object by a the customer


The SearchData document is design to model One to Squllion or better Squillion to one

One to Many


We can imagine a situation where by on the end of the purchase the CustomerData will be updated adding the concluded CartID the code below shows how this would be done in Jolie


Of course this code is simplified and it does not consider any exceptions compensation or business logic consideration, but it aims to show how is possible to manipulate MongoDB ObjectID across collections using Jolie


One to Squillions

Although the used data model is probably not the most correct to keep track of the navigation history it is a simple example to give the idea how handle this problem in Jolie 


Of course one should consider now how to search the data of this potentially enormous collection and how efficient this search may be but it is not topic of this post.

Wednesday 3 May 2017

Implementing IT solution for customer loyalty program using a microservices approach

General view

We are all familiar with the concept of  loyalty program, and probably we all are members of at least of one or two of these programs. Designing and implementing a reliable and flexible IT solution, that handles such program, may result more complex that expected.
In the recent few months I had to coordinate a small team of developers in the design and development of  such a solution for the company I work for .
The company that I work for operates in the media industry with and in focus on newspapers both classical and online.  
From the start it was evident that the implementation of this project presented the following constrains :
  • It needed to operate in well establish IT ecosystem 
  • It needed to be capable of adapting itself to the new marketing demands
  • It needed to be able to communicate and exchange data with third party software
It is my opinion that to match all three contains the best approach is  the microservices one , with the following  microservices based architecture is described by the diagram  :  




External API Service

This service provides access to data regarding the loyalty program and marketing promotion and other customer management operation. The API have been published as REST API , being REST a simpler meteorology of interaction with third parties software.

WEB Server

The solution provides the user with a website where to check is loyalty program status and interact with the marketing department The instance of  the WEB server was generated  from the  Jolie Leonardo WEB Server template and evolved to fit with the project requirements

Integration with third part Web CMS

One of the project requirement was the integration with the the Applix Polopoly WEB CMS , the customer web portal had to be embedded in standard web article of our on-line newspaper
The solution adopted was to spiting the technology stack in two all the presentational  files (HTML and CSS ) were deployed on Polopoly and the JavaScript provided by the Jolie WEB Server.
It was also necessary modify the HTTP message response header to handle access control issues adding in the HTTP port configuration  parameters as shown  in following code


Protocol: http{

.osc.OperationName.response.headers.("Access-Control-Allow-Origin") -> SiteOrigin;
.osc.OperationName.response.headers.("Access-Control-Allow-Headers") = "Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With";
.osc.OperationName.response.headers.("Access-Control-Allow-Credentials") = "true"

}

SiteOrigin is variable that should  be set to the location of the caller

Integration with an Authentication Service Provider 

In order to provide a secure and reliable process of authentication  Auth0 has been identified as our authentication  provider. Auth0 provides social and Gmail integration and several others interesting features.
Auth0  requires the client WEB Server to publish a REST callback operation to complete the login or registration process.
As in the case of the WEB CMS integration it was required to define specific HTTP message response hearder value to handle HTTP handle access control issues and  cookie policies adding in the HTTP port configuration  parameters as shown  in following code

Protocol: http{
   .osc.callback.response.headers.("Set-Cookie")->sid;
   .osc.callback.response.headers.("Access-Control-Allow-Origin") -> Auth0Site;
}


The variable sid is passed via the response Header and it allows to set a value for the cookie that will be used by the WEB Server to  provide user dependent data.
The cookies is recovered and piped in in the operation request value using the following code

Protocol: http{
 .osc.OperationName.cookies.AuthCode = "sid"
}

This will free the frontend developer from having to pass as value the AuthCode in the calls. and also it allows the backend developer to manage at port level and operation level the use of cookie

Technology stack 

The technology stack used in the web solution is the following

  • HTML5 /CSS
  • jQuery 
  • Jolie (server side)
The interaction between the HTML page and the backend operation is achieved via AJAX call. The implementation of all the JQuery function and it the operations request response types is generated automatically by a tool
Handling the asynchronicity of the AJAX is achieved using a event driven model. the tools includes in the generated code an specific event for each operation. 
   
function operationName(value) {
    $.ajax({
        url: '/operationName',
        dataType: 'json',
        type: 'POST',
        contentType: 'application/json;charset=UTF-8',

        success: function(response) {
            if (typeof(response.error) == 'undefined') {

                responseObj = new OperationNameResponse()
                responseObj.parseJson(response)
                var event = new CustomEvent('EventOperationNameResponse')
                event.data = responseObj
                document.dispatchEvent(event)
            } else {

                var eventError = new CustomEvent('EventGeneralError')
                eventError.data = response
                document.dispatchEvent(eventError)
            }

        },

        error: function() {
            var eventErrorHttp = new CustomEvent('EventGeneralHttpError')
            document.dispatchEvent(eventErrorHttp)
        },
        data: JSON.stringify(value.toJson())
    })
}


In this way the frontend programmer will be able to work with the the operation published by the WEB Server by calling operation or methods of classes in js


handler_EventModifyAnagraphicResponse(e){
    operationNameResponse = e.originalEvent.data
   /*   insert your code here */
}

$(document).ready(function() {
              $(document).on('EventGeneralError', handler_EventGeneralError);
              $(document).on('EventOperationNameResponse', handler_EventOperationNameResponse);
 }

also it will be able to handle the data from the call in an event handler function
The use of this tool has allowed the team to increase the efficiency of the Web Application development achieving a good separation between the different roles in the developing team  .

Business Logic Service  

This service is the fulcrum of the solution it handles the following aspects 
  •  Logic for the loyalty program
  •  Logic for the creation/update and recovery of customer data
  •  Logic for the creation/update and recovery of distribution data
Each of this aspects has generated a separated interface definition, to make make simple the separation of the implementation when required 
The business logic service acts as an coordinator  of internal and external data to achieve the desired business objective. The internal data are masked behind specific CRUD services to increase the decoupling between data and logic.

CRUD Services  

The CRUD Services mask data from existent database. The choice of developing specific services to publish internal data has been driven by the desire to make such data available to other future application.

Third Party Connectors 

Some of the data required by the application are provided by external service of the following list 
All three implementations are center of the use of a HTTP Jolie output port, such a port acts as a HTTP client .
The process of implementing a integration to a Third Party REST Service requires the following steps

  1. Definition of the interface and its operation ( it can be also a subset of the provided operation)
  2. Definition of the request and response type 
  3. Definition of the port parameters such as  Method Aliases ecc ecc
Using this simple methodology it has been possible developing the connector that now can be used for any other application it  may be needed for. 

Google Geocoding API

It was necessary to implement a geo localization process to populate the distributor and shop data with the their latitude ad longitude coordinate  In this way the marketing team will be able to add a new shop or distributor without worrying to provide the localization data.
These data will then available via web application to the customers searching for shop or distributor in his/her area.

Auth0

In the WebServer section it was mentioned the Auth0 as authentication  provider.  It was also necessary to implement the integration serverside with some of the API provided by Auth0. This has made possible recovering user and token info.

Prestashop 

It was also necessary to implement the integration with our instance of Prestashop, The business requirement was to provide  to our customer a feedback on the use of their loyalty points via our e-commence allowing the customers to see customer on run-time the status of its discount voucher and the history of his/her purchases