API Crafting Secrets: into the ConveyAPI

We did not set out to create an API. We initially developed Convey as an internal library for use in Converseon’s products and service.”

Interview with Mark Walz VP Product Management at Converseon. You can follow Mark on twitter: @rocketmark

Tell us about the development process of the Convey API
We did not set out to create an API. We initially developed Convey as an internal library for use in Converseon’s products and service. The Convey library has been several years in the making and has involved industry and academic expertise from its inception.

As Convey’s power and accuracy grew, we began to see applications for it outside of our own products and so did our clients. As received inquiries and proposal requests for large volume text analytics, we decided to take the Convey library and make it available as an API. This empowers anyone to place text analytics inside their applications, reports, tools, etc.

The API itself has evolved over the course of 6 months. It’s been a very rapid process. We use the typical ‘agile’ tools (Jira, Github, Skype, instant messaging, etc). We don’t follow any one specific flavor of ‘agile’. In general we are very pragmatic: a reasonable number of tickets at any given time; responsible developers who are empowered to make decisions; test but don’t over test; review often, deploy often. Too much process or methodology becomes cumbersome and counter productive.

What are the key components of your core API infrastructure?
3scale is obviously a big part of our API infrastructure ☺. Being able to address so many little details so quickly has been phenomenal. Behind the 3scale curtain is a JVM backed API that scales across a cloud infrastructure. Our Convey text analytics engine processes large volumes of text and outputs annotations in milliseconds. It’s as near to real time as text analytics can get.

What drove your choices in terms of technology and infrastructure?
There are the usual concerns: choosing a language that did not tie us to a specific operating system, performance, reliability, maturity, a hosting platform with flexibility, a strong developer eco-system, etc. All these factors played into our decisions. Beyond those basics, speed, agility and flexibility were key.
We need to have the power to develop and iterate quickly. The JVM gives us that power. We can use Java, Scala or Clojure and reliably deploy our code. We can also deploy it to a number of different cloud infrastructures. AWS, Rackspace, internal environments and other scenarios are not a problem. When combined, we are able to quickly scale in a number of different ways.

3scale provides the easiest and most cost effective path to authentication, usage, test harnesses, etc. Taking care of all of that, essentially overnight, let us focus on our core products and maintain a healthy, innovative pace.

What are the specificities of your API?
The API itself is very simple. In it’s most basic form, it requires a key and a block of text. It takes the block of text and provides a number of annotations for that text. The API can also take parameters to turn on and off the annotations and specify the level of detail to analyze (document, sentence or entity).

Results are available in JSON or JSONP. Each result set contains a ‘Status’ area that has some additional information. We’ve tried to make it as simple as possible to use the power of Convey’s advanced text analytics capabilities.

How can developers get a better taste of what your API does?
3scale ActiveDocs for the Convey API provides the easiest way to test and explore the API. It’s one of the reasons we chose 3scale. We also have online documentation available within the 3scale backed portal and a Github account with some code examples and how-tos that work immediately (JavaScript examples that run locally in a browser with no compilation or installation).

Any roadmap/evolution plan that you can share with us?
ConveyAPI is being continually enhanced. Immediate improvements include several new annotations we are working hard to complete. Along with the new annotations, we have many other enhancements:

  • Support at least 10 more languages by the end of 2012

  • Extension of the API to include methods for bulk processing (as opposed to single record submission)

  • Socket level support for some API calls

  • vMethods for pushing back information regarding annotations (i.e. the ability to correct an annotation)

  • The ability to build your own sets of trained data (stayed tuned on this front!)

ConveyAPI by ConverseonConveyAPI™ is a simple to use REST web service that provides programmatic access to a powerful text analytics engine. It utilizes natural language processing, statistical modeling and machine learning techniques to return content-specific annotations, including Sentiment, Emotion, Intensity & Relevance. ConveyAPI’s annotations yield near-human accuracy and can scale as wide as you need with respect to API calls.