Natural Language Processing: Quick Win for Digital Transformation

Join Thad for an overview of Natural Language Processing on March 9th at Data Driven Charlotte! Register here: https://register.gotowebinar.com/register/8542813235093174800?source=Socials

As a fellow data and IT leaders, you know a few things about the mystical journey of “digital transformation”:

1. PULL - You have loads of under-used data. New tools and cloud technology empower you to put that data to use.

2. PUSH - Competitors and disruptive startups are already on the “digital transformation path.” Many have used COVID to break barriers and accelerate innovation.

3. DATA - “DT” should change processes and products fundamentally, by bringing data closer to where the money is, usually in customers’ hands.

4. LOTS OF TALK - There are swarms of white papers and buzzwords.

5. YOU ARE THE ACTION - Leadership turns to us data and IT leaders to know what this digital transformation is and how to get there.

Quick Win for Digital Transformation

Text data, such as emails, customer contacts, complaints, social media, contracts, etc., might be your fast win in “digital transformation.”

• Text / voice data is close to value: it’s untapped and it’s often from customers

• It’s easy to access. Applications like content management, ticketing, or ERP store raw text. Natural Language Processing (NLP) libraries clean and use raw text. There’s no need to warehouse or even model the data

• It’s expensive (without NLP). There’s usually no alternative to use text data except having expensive humans read it

• NLP techniques are fairly new, so you can still get a jump on competitors

You could turn text data into value, for example, by

• Making sense of what customers are telling you in social media, product reviews, and other feedback. Or even scanning that for sales and service opportunities.

• Building a chatbot that makes life easier for customers (and cheaper for your contact center)

• Build better sales tools - scripts, training, coaching - by analyzing customer / salesperson conversations

• Scanning contracts for risky language or lack of certain language

• Checking that product descriptions on your e-commerce website each contain certain elements (size, color, usage, quality), and flagging those that need a human to review and rewrite them

Let’s take a quick look at how NLP has become feasible and effective just over the last few years. Then, we’ll look at the implementation path, to get you thinking about where NLP might create quick value for you.

Recently Feasible

A wonderful reason that NLP is feasible is that there is very little data preparation work required. No data warehouse, no data modeling, no table structures. It’s usually an easy step to extract text from the CRM, content management system, ticketing system, etc.. These note from a service representative were simply in the “rep_notes” field of a contact management system:

“Customer inquired if issues with system as they seem to be experiencing delays. Confirmed there was a reported delay in confirmations but that it was not affecting order processing. Logistics is researching individual orders separately to ensure status. Customer was ok with this information.”

Open source libraries have a closet full of cleaning functions to make text usable for NLP. They remove irrelevant words (the, and, of, etc), remove or substitute overly common words (like ‘customer’ and your company name), and condense different forms of words to their root. That leaves just the substantial words:

“Inquire issue system seem experience delay confirmation report delay confirmation affect order process logistics research individual order separately ensure status ok information”

The biggest leap in NLP has been turning text into numeric data that a computer can work with. Until a few years ago, you could make text into numeric data by assigning one column to each word and counting the occurrences of each word. Then you still had thousands of columns even for a small body of texts. That is not very useful from a machine learning standpoint, and the machine doesn’t recognize the similarity between words: tire, tread, wheel, flat and rubber are related to each other, and so are piston, ring, compression, displacement, and motor. And depending on context, “oil” could be an engine component or something on the road that causes tires to lose traction.

“Embeddings” were invented over the past few years by Google, Facebook, Stanford University, among others. Their clever ideas was to notice which other words appear before and after a given target word, with the idea that “You can tell a word by the company it keeps” or “Words of a feather flock together.” Then they use a light neural network to turn those context words into a vector, a series of numbers unique to the target. Here is the word “feather” according to Standford’s GloVe library.

What’s amazing is that we can actually add and subtract these series of numbers. Now a computer can do what’s intuitive for us:

King - man + woman = ??

Truck - cargo + passenger = ??

The examples are mostly for fun, but you can see how encoding text as numbers helps the machine “understand” text by comparing these numbers. If I type in my email:

Hi Shauna, When you get a chance, can I ask you to ______

… eat more broccoli ?

… read this presentation ?

Unless you’re from the Broccoli Growers’ Association of America or you’re Shauna’s nutrition coach, you can choose the more likely ending.

Putting Text Data to Use

By recognizing these patterns in words, you can ask a machine to help provide better customer service

• compare whether a new piece of customer feedback is more like feedback about your prices or your service or your products.

• take a customer chat entry and tell you which common customer questions that entry is most similar to… the engine of a chatbot

• Analyze email or customer phone calls to find words and phrases that are buying signals, and build those into sales tools, sales training, and FAQs. Here’s a great example of that by Gloor and company at MIT: https://hbr.org/2019/03/a-novel-way-to-boost-client-satisfaction

Those examples should get your mental gears turning. What text do you have? What value might be in it?

So, let’s look at the implementation path, which can be quite short given that the pre-processing is minimal.

1. THE IDEA - If we’re going to make digital transformation happen, the initiative has to come from us in data and IT. Bounce an idea off of several stakeholders first, so you have an informal team of “investors.”

2. THE QUIET PILOT - It is often more successful to do a limited pilot under the radar. NLP is great for that because you don’t have a lot of expense in the data prep phase and you can get a working prototype quickly.

3. THE PITCH - With a realistic estimate of benefit and some vocal fans from the pilot, pitch the idea to get executive support.

4. SME INVOLVEMENT - Good work in digital transformation and data science will always change someone’s world. So get those front-line people involved and excited from the beginning. One of the best ways is to mock up the application that the work will plug into - even if it’s just a spreadsheet tool or some wireframe drawings. Then ask front-line people how they’d modify the mock-up to make their job easier.

5. PREPARATION - The data gathering for NLP is short and sweet! Usually you just need access to the source application that houses text data.

6. DATA SCIENCE / DEVELOPMENT - Highly developed open source libraries do a lot of the work, though you’ll need a data scientist with some experience to get a good result. Chatbots come with nearly all the data science hidden, though a data scientist who understands model training is still an asset. And a mere PC with graphics processor may churn for several hours but is sufficient for most model development. If there is some application development to be done, you can do that in parallel since you know what the output will look like from your SME input.

7. MVP PILOT - A “minimum viable product” is vital in data science projects. Whereas in an IT project, a pilot helps you fix bugs and find user stories you might not have foreseen, for data science projects, the MVP Pilot tackles an added uncertainty: you cannot be sure how accurate the models will be in practice so you need a chance to re-tune them.

8. MEASURE ACCURACY AND BENEFITS - After a pilot, you’ll measure the accuracy of models and the benefit of outcomes. And you’ll re-tune the NLP models using new data. Did the chatbot reduce call volume, and how often did it just add a step for frustrated customers? Was the classification of user feedback into reasons accurate vs a manual check, and how did we change processes to react?

9. MAINTENANCE - Not just after the pilot, but on a quarterly or semi-annual basis, you’ll re-tune the NLP models and measure their accuracy. There might be …. new problems appearing in customer feedback. …. process changes a chatbot needs to catch up with … etc. So plan for some degree of maintenance.

“Digital Transformation” is a happy buzzword until we in data and IT make it a reality. Natural Language Processing projects can be a quick win, because they make value out of unused data without a lot of data preparation overhead. There are many applications, and most have immediate value because they are close to customer interactions where the money is made. We’re always happy to take a look at your situation and help spark some ideas you can pitch and pilot.