17
Nov 17

NLP with AIML - Part 1 - Stopwords

Folks learning about chatbots and AIML are likely to ask if AIML is a Natural Language Processing (NLP) implementation. The answer is no. However, can one use AIML to perform NLP functions? ABSOLUTELY! It takes some work and creative use of recursion and predicates, but it can be done.

This should be the first in a series of posts that describe how you can perform NLP tasks with AIML. In this post, we'll tackle a very simple concept: removing stopwords.

In NLP, there is the concept of a pre-processing step whereby very common (and potentially meaningless) words are removed. Examples of these words are: is, the, and, I, will, just, so, than... So an input of "this is a simple sentence" becomes "simple sentence."

In most programming languages, we can conceive of looping through and testing each word. AIML doesn't naturally have this feature (ok, technically in AIML 2.0, there is a tag called , but with the exception of the example in the standard which involves counting numbers, I can't see how to use this in other ways - please comment if you have good examples of ). But we can force AIML to loop using reductions... the tag. The part where we really need to get creative is how we stop the loop. In this case, I basically have a test for when we're at the end.

At this point, I think it's better to show than tell...

Here's a flowchart of the method (no criticizing my flowcharting skills... but please ask questions!):

Flowchart for stopword removal in AIML

And even better than that, you can see this in work in two ways:

1) If you have an account on the pandorabots playground ( playground.pandorabots.com ), search in the clubhouse for "NLP Test Bot" and you can chat directly. Here's a screen shot of a chat:
Chat window with NLP Test Bot

2) I have a zip file of 3 AIML files that you can play around with yourself. These are the same files as what is loaded in the NLP Test Bot on pandorabots playground. A couple things to know... the list of stopwords comes from python's nltk library. It seemed like a good list. However, given that in AIML, we normally expand contractions, half of this list could probably be removed. The other thing to know is that it's AIML v2.0. That's because I am using the and tags.

Download nlpinaiml.z7

2a) Could this be done with AIML v1.0? Yup. Before the and tags came into existence, I had a category that looked something like this:

<pattern>SPLITME * *</pattern>
<template>
<think>
<set name="firstword"><star index="1"/></set>
<set name="remainder"><star index="2"/></set>
</think>
</template>

 

There was also a second category: SPLITME * which, when both of these existed, would match if * only had one word.

Hopefully I'll get around to posting some of the next steps... like part of speech tagging (hint: I do a lot of the same stuff with reduction and setting predicates... the trick is to have a good base of words) and then what really matters: how you can use this to have an interesting chatbot!

Please comment/ask questions...

28
Sep 16

AIML + Python + Python Libraries = I can talk with my chatbot

It's been a while...

...and I've recently decided to continue to work with AIML.

Why? Python. My newest programming love. I shunned it at first. I mean... you have to maintain the same amount of spaces?? That seemed ridiculous when I first head about it so many years ago (I'm a very old-school programmer). But I've been forced to use it and now that I have... it's so easy, which translates to the amount of time needed to get something done. Time, these days, is my most precious resource.

Which is why when I had an idea for a chatbot that involved speaking and listening to it, I wondered how easy or difficult it would be to do in python. After a few minutes of Googling, I saw that the three components I needed to make this happen all existed: PyAIML, a speech recognition library, and a text-to-speech library.

How long did it take to put it all together so I could speak to Zoe?

Less than an hour. And it only took that long because I kept getting interrupted by  two impatient guys wondering where dinner was my loving family.

It was so easy, I almost didn't think it was worth blogging about. But I figured that some might need this guide, so here it is. How to talk to your chatbot. All you need is some AIML (version 1.0.1... the PyAIML I'm using doesn't (yet) support AIML 2.).

Install the following:

pip install aiml
pip install SpeechRecognition
pip install PyAudio
pip install pyttsx

A couple notes... first, if you need more help on the PyAIML, look to this very helpful post: http://www.devdungeon.com/content/ai-chat-bot-python-aiml

If you don't already have it, you might nee pywin32 from sourceforge:  https://sourceforge.net/projects/pywin32/

And then here's what you python file looks like with it all put together:

import aiml
import speech_recognition as sr
import pyttsx
import os

# Create the kernel and learn AIML files
kernel = aiml.Kernel()
if os.path.isfile("bot_brain.brn"):
     kernel.bootstrap(brainFile = "bot_brain.brn")
else:
     kernel.bootstrap(learnFiles = "zoe-startup.xml", commands = "load zoe")
     kernel.saveBrain("bot_brain.brn")

# Start the TTS engine
engine = pyttsx.init('sapi5')
voices = engine.getProperty('voices')

# obtain audio from the microphone
r = sr.Recognizer()

# Press CTRL-C to break this loop
while True:
     # obtain audio from microphone
     with sr.Microphone() as source:
         print("Say something!")
         audio = r.listen(source)
     try:
         myinput = r.recognize_google(audio)
     except sr.UnknownValueError:
         print("Google Speech Recognition could not understand audio")
     except sr.RequestError as e:
         print("Could not request results from Google Speech Recognition service; {0}".format(e))

        print "You said: ", myinput
     if myinput == "exit":
         exit()
     # Get Zoe's response
     zoes_response = kernel.respond(myinput)
     print "Zoe said: ", zoes_response
     engine.setProperty('voice',voices[1].id)
     # have Zoe say the response
     engine.say(zoes_response)
     engine.runAndWait()

 

Of course, you'll need your own startup.xml file and corresponding aiml files (refer back to that helpful post I mentioned on PyAIML). And I chose the female voice when I set voices[1].id. On my windows machine, pyttsx only has one male and one female voice to start with.

Happy chatting!

Xkcd captured Python perfectly:  https://xkcd.com/353/