Sometimes it is hard to discover if a conversational partner on the other end is a real person or a chatbot. In fact, it is getting harder as technology progresses. A well-known way to measure the chatbot intelligence in a more or less objective manner is the so-called Turing Test. This test determines how well a chatbot is capable of appearing like a real person by giving responses indistinguishable from a human’s response.
Some bots communicate with other users of Internet-based services, via instant messaging (IM), Internet Relay Chat (IRC), or another web interface such as Facebook Bots and Twitterbots. These chatterbots may allow people to ask questions in plain English and then formulate a proper response. These bots can often handle many tasks, including reporting weather, zip-code information, sports scores, converting currency or other units, etc.[citation needed] Others are used for entertainment, such as SmarterChild on AOL Instant Messenger and MSN Messenger.

This is where most applications of NLP struggle, and not just chatbots. Any system or application that relies upon a machine’s ability to parse human speech is likely to struggle with the complexities inherent in elements of speech such as metaphors and similes. Despite these considerable limitations, chatbots are becoming increasingly sophisticated, responsive, and more “natural.”

2a : a computer program that performs automatic repetitive tasks : agent sense 5 Several shopping "bots" will track down prices for on-line merchandise from a variety of vendors.— Sam Vincent Meddis especially : one designed to perform a malicious action These bot programs churn away all day and night, prodding at millions of random IP addresses looking for holes to crawl through. — Jennifer Tanaka
The first formal instantiation of a Turing Test for machine intelligence is a Loebner Prize and has been organized since 1991. In a typical setup, there are three areas: the computer area with typically 3-5 computers, each running a stand-alone version (i.e. not connected with the internet) of the participating chatbot, an area for the human judges, typically four persons, and another area for the ‘confederates’, typically 3-5 voluntary humans, dependent on the number of chatbot participants. The human judges, working on their own terminal separated from one another, engage in a conversation with a human or a computer through the terminal, not knowing whether they are connected to a computer or a human. Then, they simply start to interact. The organizing committee requires that conversations are restricted to a single topic. The task for the human judges is to recognize chatbot responses and distinguish them from conversations with humans. If the judges cannot reliably distinguish the chatbot from the human, the chatbot is said to have passed the test.
“We believe that you don’t need to know how to program to build a bot, that’s what inspired us at Chatfuel a year ago when we started bot builder. We noticed bots becoming hyper-local, i.e. a bot for a soccer team to keep in touch with fans or a small art community bot. Bots are efficient and when you let anyone create them easily magic happens.” — Dmitrii Dumik, Founder of Chatfuel