Giancarlo Frison Signals from the Noise

Machine Comprehension on Chatbots

One of the most demanded feature in chatbots is the ability to automatically provide helpful informations. Users might ask about how to pay the purchases online, how to return an defected item, when the purchase could be delivered or just about the opening hours of a shop.

One way to implement this feature is to train a sentence classifier for a determined set of questions the merchant is willing to answer. The system should be instructed on some examples such as: “which credit card do you accept?”, “How do I pay?”, “which payment do you support?” and so on. This simple technique requires a sequence of manual tasks for every conversational agent, such as set up the training and inference pipeline for questions/answers, or reuse the Natural Language Understanding (NLU) system already adopted by the chatbot, if present.

see full article

SOA example application

SOA describes a set of patterns for creating loosely coupled, standards-based business-aligned services that, because of the separation of concerns between description, implementation, and binding, provide a new level of flexibility.

Service Oriented Architecture terminology has spread in recent years, at least among people who were involved in most of the Information Technology activities. The guidelines suggested by this methodology are granted as major factors to succeed in different distributable systems domains. Just as the definition is clear and easy to understand, so is its implementation into a real project, being intuitive, concise and elegant.

I have released an application demonstrating how SOA’s principles can be applied into a small project making use of EIP (Enterprise Integration Pattern), IoC (Inversion of Control), and a building tool and scripting language such as Groovy. I analized a simple business case: an entertainment provider who wanted to dispatch rewards and bonuses to some of its customers, depending on customer service’s subscriptions. The process sequence is simple:

It is required to provide an implementation of a RewardsService. The service accepts as input a customer account number and a portfolio containing channels subscriptions. The Customer Status team is currently developing the EligibilityService which accepts the account number as an input.

I set up an infrastructure to write acceptance tests for this first meaningful feature. This is what could be defined as a “walking skeleton,” a prototype with the essential aspect that it could be built, deployed and tested after being easily downloaded from Github.

RewardService is invoked by the client and it calls, in turn, the eligibility service which however, in this case is not implemented. As many real scenarios expect external services, this proof-of-concept refers the eligibility service to a black-box, where only request/response interface is known.

The unit test simulates the eligibility service behaviors mocking the end-point through the Camel Testing Framework. However, if you want to run the application on your local machine I set up, within a line of code, a faux eligibility service that merely returns a positive response:

def alwaysEligible = {exchange ->
  if(exchange){
    exchange.getOut().setBody('CUSTOMER_ELIGIBLE')
  }
} as Processor

The entry point is an HTTP Restful interface built upon the Apache CXF, and is easily set up within few lines in the configuration. CXF is initialized by Spring in this following way:

jaxrs.'server'(id:'restService',
  address:'http://${http.host}:${http.port}') {
    jaxrs.'serviceBeans'{ ref(bean:'rewardService')}
}

Services are connected by Apache Camel. RewardService contains only the reference of the ESB context – an instance of ProducerTemplate. Such solution allows a complete separation between the linking system and the business services. The Camel context represents the SOA’s wiring, and is configured through a DSL as in the example below:

from('direct:rewards').to(eligibilityServiceEndpoint)

Gradle archetype for Spring applications

I am releasing a Gradle archetype useful for creating Java/Groovy applications based on Springframework. Of course, it is not a real archetype because such a creation is not possible. However, with very few steps you can create, edit and deploy an application server. It would be a most accommodating starting point for deployable software projects.

This release is an attempt to mitigate common issues related to development life-cycle phases such as testing, the running of application and deployment in various environments. The archetype leverages upon the flexible building process and on the top-most featured IoC (Inversion of Control) management system.

When creating application modules for linking services through HTTP, JMS or any other connector type, this archetype is refined and can be applied for satisfying these requirements:

  • Automatic testing, building and continuous integration.
  • A different configuration for each environment (development, integration, production).
  • Springframework based system.
  • Groovy support.

The project consists of:

  • Utility classes for given Spring context
  • Grails-like DSL for Spring setup (beans.groovy).
  • Logging and application configuration properties for each environment (development/integration/production).
  • Gradle config file.

Why Gradle?

Problems exist using Maven in Groovy projects due to the gmaven plugin, which may indicate that it is not ready for the groovy-user community. Indeed, Gradle works perfectly on Groovy projects. It is so concise and elastic that you don’t have just a building system, you have a programming tool. When a customized behaviours proper plugin cannot be found in the registry, you may add custom tasks by writing groovy code directly to the build.gradle descriptor. Gradle is a swiss army knife for developers.

Getting started

  • Run
git clone [email protected]:gfrison/proto-app.git myApp

where myApp is the name of your project.

  • Edit property ‘projectName’ in ‘build.gradle’ with project name.
  • Add classes, and manage them with spring ‘beans.groovy’.
  • You are now ready to test, run and deploy your project through a continuous integration system such as Jenkins.

If you have suggestions, or pull requests from Github, myself the author, would be happy to consider them.

Software architect mistakes

I think that to get up in the morning and brew a good cup of coffee is one of the best way to start the day. You know, the heady fragrance that emanates from the machine-pot, it’s delicious. When it’s ready, pour the coffee into a cup, add some sugar, and finally you got it – end of the coffee making process.

Have you ever thought to design a coffee making process with some diagrams, or doing the same with other banal activities such as taking a shower? Of course not. For other cases less trivial than these, including software project development, a minimal-design work can be quite useful and somewhat needed. Often questions arise; is an architecture design worth the time and effort invested in it? Well, you may answer this question first: Are there risks in the project that could be minimized by an early design activity?

The more ambitious and challenging the project is, the higher the number of risks, and the more difficult it is to complete successfully.

How to identifying risks

The easiest place to start is with requirements, in whatever form they take, and to look for things that seem difficult to achieve. Gathering requirements is fundamental for deciding what to do and how. However, sometimes problems arise at this starting point that lead to the ruination of the project. Some assumptions may underestimate this key phase and shake the architect role to its foundations:

It’s someone else’ responsibility to do requirements

Domains drive the architecture choices, not vice-versa. Requirements can create architecture problems. At the very least, you need to assist the business analysts.

I learn the domain as I write the code; incrementally

While prototyping pieces of software is a way for mitigating engineering risks and figuring out the hardest problems, writing code could be a waste of time for analyzing a domain. Rather, it’s very cost-effective to modelling it in advance.

The requirements are already fully understood by the stakeholders

Clear communication is critical between people and the role of a software architect can be a very difficult one when others don’t understand what you do and why.

Domains are irrelevant to architecture choice

Developers may copy an architecture from a past project. Maybe just following the company standard, but ignoring the motivations behind previous choices. They are more likely to be unaware of the qualities required in the current project.

I already know the requirements

At least the documentation should be in your mind, but designers should use models to amplifying their reasoning abilities and unfold not clearly visible aspects that affect their own risks.

Cool DSL with Groovy

Domain model design has never been confused with ‘ease’. From the dawn of its conception, generating executable Unified Modeling Language (UML) diagrams meant sweat and frustration. Generating stubs & skeletons, alone, led one into quagmires of Java Enterprise Edition. Yet it is inherently possible — and actually has been for ages — to design a programming language dedicated to solve specific domain problems; effortlessly and quickly.

DSL is not a foreign thing. Developers are steeped in DSL, even if unwittingly. Frameworks are DSL. A macro is also DSL. When you write a function? That too is steeped in DSL. But functions are often a dirty business; un-standarized; quirky, whereby even domain experts face a daunting task when unraveling and then being put to task to update such rapscallion boilerplates (that is if they do not themselves compound the problems). A programmer worth his reputation would find it necessary to isolate and clean up all non specific domain terms within the DSL. This literally means rooting through and setting aside all secondary-in-importance setup code, allowing the domain expert to establish or reestablish the primary foundation code. This is most times done for large money for an end user; the helpless customer.

Given an appropriate DSL that fits their needs, customers could write all of the code that they need themselves, without having to be programmers.

Now let us imagine, or better, take for granted, that a good DSL experience can exists. That given an appropriate and standardized DSL environment, even the customer, with a few simple tools and grasp-of-concept, can self-write code, gracefully, and specific to their needs without the cost and hassle of having to become, or running to, programmers. And that is where Groovy comes in.

What’s Groovy?

It’s Java as it should be. Easy and intuitive, it offers new features unknown to its parent yet (I’m still waiting for release 7), and come up with those benefits that form the basis of the DSLs that we will develop.

A fundamental feature is the MOP, the ability of changing runtime the properties and the behaviour of objects. It allows us to respond to method calls that do not exist in the class, in other word to “pretend” that these methods exist. Another essence to consider is the Closure. It’s the real power of Groovy. The extreme flexibility of this kind of object/method allows us to change its behaviour replacing its delegate class, on the fly.

Closure delegate

Groovy has three variables inside each closure for defining different classes in its scope: this, owner, and delegate. The this variable refers to the enclosing class. The owner variable is the enclosing object of the closure. The delegate variable is the same as the owner, unless that delegate is substituted.

class MyClass {
  def closure = {
    println "this class:"+this.class.name
    println "delegate class:"+delegate.class.name
    def nested = {
      println "owner nested class:"+owner.class.name
    }
    nested()
  }
}
def tryme = new MyClass().closure
tryme.delegate = this
tryme()

//output:
//this class:MyClass
//delegate class:Script1
//owner nested class:MyClass_closure1

When a closure encounters a method call that it cannot handle itself, it automatically relays the invocation to its owner object. If this fails, it relays the invocation to its delegate. one of the reasons builders work the way they do is because they are able to assign the delegate of a closure to themselves.