Research Study
Do Website Chatbots Actually Turn Visitors Into Customers? What the Research Says
Published November 15, 2026 · Research by Cliff Tillery
Abstract
This report reviews the public evidence on whether website chatbots turn visitors into customers, and what it means for local service businesses that win customers by phone, form and appointment rather than online checkout. The strongest evidence comes from randomized field experiments. In the largest and most recent, a 2023 to 2024 test with 44,614 shoppers on a large online retail platform, a generative AI pre-sale chatbot raised the conversion rate by 21.7 percent and sales per shopper by 16.3 percent, with a bigger gain when the bot could hand customers to a human. A 2019 experiment on outbound sales calls, not website chat, found that an undisclosed AI agent matched the best human sellers, but purchases fell 79.7 percent when the same agent disclosed it was a bot before the conversation. Customers are wary: a Gartner survey found 64 percent would prefer that companies not use AI for customer service. The widely repeated vendor figures, such as chatters being 5 times more likely to buy, compare people who chose to chat with people who did not, so they cannot show that the chat caused the sale. No published controlled study of chatbots on local service business websites was found, so these results may be larger or smaller for a contractor or clinic. A chatbot is worth testing for after-hours leads and routine questions, as long as it hands off to a human, never invents answers, and is measured on booked calls and appointments rather than chats.
Methodology
This report was prepared by Make It Loud Digital Marketing. We searched academic databases, preprint servers and industry sources in November 2026 for evidence on whether website or messaging chatbots change purchase or conversion rates. We prioritized randomized field experiments with real customers, because they are the only studies that can show cause and effect, then peer-reviewed observational studies, large customer surveys and legal decisions. Figures come from the original papers and releases where available. Vendor statistics are reported as compiled by a chat-software vendor and were not independently verified. Make It Loud's contribution is review, synthesis and interpretation of published evidence, not a new experiment or primary data collection, and this report should not be read as a proprietary Make It Loud study of chatbot performance.
Findings
A well-built chatbot can turn more visitors into customers, but the best evidence comes from large online stores, not local service businesses, and the details matter more than the widget itself. In the most rigorous public test, 44,614 shoppers on a cross-border e-commerce platform, an AI pre-sale chatbot raised conversion 21.7 percent and sales per shopper 16.3 percent, and the gain was larger when the bot could escalate to a human. How a bot introduces itself, what language it speaks and whether it can resolve the problem swing results in both directions: disclosing a bot before an outbound sales call cut purchases 79.7 percent in a 2019 experiment, while adding a second language to a shopping chatbot raised daily purchases about 132 percent. Customers are cautious, with 64 percent telling Gartner they would prefer companies not use AI for service. The eye-catching vendor figures compare people who chose to chat with people who did not, so they cannot establish causation. For a local business, the research points to accuracy, a clear human handoff and honest measurement as the factors that decide whether a chatbot pays off.
Overview
What the controlled experiments, customer surveys, legal decisions and vendor claims actually show about whether chatbots raise conversions, and what it means for local businesses that win customers by phone, form and appointment.
Chat widgets and AI assistants are now standard offers from website builders, scheduling platforms and marketing software, and business owners are often told they will double conversions. Most of the statistics behind those claims come from the companies selling the software. Independent evidence is harder to find, and it rarely involves the kinds of businesses that make up most of the local economy: plumbers, dentists, med spas, HVAC contractors, attorneys and auto shops, whose customers usually convert by calling, filling out a form or booking an appointment rather than checking out online.
This report reviews what the public research actually shows. We focused on randomized field experiments, in which real customers are randomly assigned to see a chatbot or not, because they are the only studies that can show cause and effect. We also summarize large customer surveys, a peer-reviewed study of thousands of real chatbot conversations, a legal decision on chatbot accuracy, state disclosure laws and the most widely repeated vendor statistics, then translate the findings into practical guidance.
A few cautions are worth stating up front, because the headline numbers are easy to misread. The 21.7 percent conversion lift comes from an online retail platform, not a local service business, so it should not be assumed to apply to a contractor or clinic. The 79.7 percent drop came from outbound sales phone calls, not website chat, so it is not a universal verdict on website chatbots. And the vendor figures compare people who chose to chat with people who did not, which is a biased comparison that cannot establish cause. This is a review of outside research, not a Make It Loud experiment.
Download the full report
You can download the full research report as a PDF. It is a review of published evidence, prepared by Make It Loud Digital Marketing and dated November 2026, and it reproduces the findings, tables, methodology, limitations and sources below.
Key findings at a glance
| Measure | What the evidence shows |
|---|---|
| Best controlled test | A 21.7 percent conversion lift from an AI pre-sale chatbot, on an online retail platform, among 44,614 shoppers |
| Human handoff | Sales were 25 percent higher when the bot could escalate to a person than with no pre-sale service at all |
| Disclosure effect | Disclosing a bot before an outbound sales call cut purchases 79.7 percent in a 2019 experiment |
| Customer attitudes | 64 percent of surveyed customers prefer companies not use AI for customer service |
| Vendor claims | Figures like 5 times more likely to buy compare choosers with non-choosers and show no causation |
| Local service evidence | No published randomized study of chatbots on local service business websites was found |
Sources are listed in full at the end of this report.
Finding 1: The Best Controlled Test Found a 21.7% Lift in Conversion
The largest and most recent randomized test of a website sales chatbot was conducted by researchers from Columbia University and Zhejiang University with a large cross-border e-commerce platform between September 2023 and June 2024. Shoppers were randomly assigned to have access to a generative AI pre-sale chatbot, available 24 hours a day, or not. Among 44,614 consumers, the chatbot raised the conversion rate by 21.7 percent and sales per shopper by 16.3 percent, both statistically significant. Customers’ post-purchase ratings also rose about 4.5 percent.
Two details matter for small businesses. First, the chatbot performed best when it could escalate to a human: with escalation, sales were 25 percent higher than with no pre-sale service at all, and customers in that arrangement spent about 11.5 percent more than those served by human agents alone. Second, the gains were concentrated among newer and less experienced customers, the people who most need answers before they buy. One caveat is central: this was an online retail platform where customers check out on the site, so the size of the lift should not be assumed to carry over to a local service business where people convert by phone or appointment.
| Effect of an AI pre-sale chatbot (44,614 shoppers) | Result |
|---|---|
| Sales vs. no pre-sale service (chatbot with human escalation) | +25.0% |
| Conversion rate (chatbot vs. no chatbot) | +21.7% |
| Sales per shopper (chatbot vs. no chatbot) | +16.3% |
| Spending vs. human agents only (chatbot with escalation) | +11.5% |
| Post-purchase rating | +4.5% |
Source: “Generative AI and Sales Productivity: Field Experiments in Online Retail” (Columbia University, Zhejiang University and Zhejiang University of Finance and Economics; arXiv:2510.12049, 2025). Experiment run September 2023 through June 2024 on an unnamed cross-border e-commerce platform.
Finding 2: How a Bot Introduces Itself Can Make or Break the Sale
A widely cited 2019 study in Marketing Science by Xueming Luo, Siliang Tong, Zheng Fang and Zhe Qu tested AI voice agents against human sales staff at a large financial services company, using 6,255 randomly assigned customers who were offered a loan renewal. When customers were not told they were speaking with a bot, the AI agent’s purchase rate of 23.7 percent was statistically the same as the company’s best human sellers at 25.1 percent, and nearly five times that of its weakest at 4.9 percent. But when the bot disclosed that it was AI before the conversation began, purchases fell to 4.8 percent, a drop of 79.7 percent.
This study involved outbound phone calls, not website chat, so it is not a direct measure of website chatbots, and the 79.7 percent drop should not be read as a universal verdict on chat widgets. Its lesson is about customer expectations: people judged the disclosed bot as less knowledgeable and less empathetic even though it performed the same. That creates a tension for businesses, because a growing number of laws require bots to identify themselves. California’s B.O.T. Act, in effect since 2019, requires clear disclosure when a bot is used to encourage a sale, and Utah requires businesses to disclose generative AI when a consumer asks, and up front in certain high-risk interactions. The practical answer is to disclose and then perform well enough to overcome the skepticism.
| Purchase rates by agent type (6,255 customers) | Purchase rate |
|---|---|
| Top 20% human agents | 25.1% |
| AI agent, not disclosed | 23.7% |
| AI agent, disclosed after the conversation | 11.0% |
| Bottom 20% human agents | 4.9% |
| AI agent, disclosed before the conversation | 4.8% |
Source: Luo, Tong, Fang and Qu, “Frontiers: Machines vs. Humans: The Impact of Artificial Intelligence Chatbot Disclosure on Customer Purchases,” Marketing Science 38(6), 2019. Outbound loan-renewal sales calls at a large Asian fintech company.
Finding 3: Language and Tone Matter, but So Does Getting the Basics Right
Smaller experiments show that a chatbot’s design can change results substantially. In a randomized test published in the Journal of Business Research in 2025, a chatbot-based shopping app in India added Hindi alongside English for a random group of 3,952 new users, compared with 8,683 who saw English only. Average daily purchases rose from 0.025 to 0.058 per user, about 132 percent, with the gains concentrated in higher-involvement categories such as travel and local deals. But the bilingual version also saw sharply higher uninstalls, a reminder that more engagement is not always better engagement.
A 2025 field experiment on a U.S. travel-planning site tested the tone of a GPT-4o assistant with 282 users. Users who chatted with an assistant given explicit tone instructions, whether enthusiastic or neutral, subscribed at roughly 14 to 15 percent, versus about 5.5 percent for an assistant given no tone guidance. The sample was small and the outcome was subscriptions rather than purchases, but the direction is consistent: the instructions and setup behind a chatbot matter, not just whether one exists. For metro Atlanta businesses serving Spanish-speaking customers, the language finding is especially relevant.
Finding 4: Customers Are Wary, and a Bad Bot Can Cost You the Sale
Customer attitudes are a real constraint. In a Gartner survey of 5,728 customers conducted in December 2023, 64 percent said they would prefer that companies not use AI for customer service, and 53 percent said they would consider switching to a competitor if a company did. Their top concern was that AI would make it harder to reach a person, followed by fears of job losses and wrong answers. In September 2026, Gartner reported that only 27 percent of customers would try a chatbot again after a negative experience.
Peer-reviewed research shows why. A 2022 Journal of Marketing study by Cammy Crolic, Felipe Thomaz, Rhonda Hadi and Andrew Stephen analyzed about 35,000 real chat sessions at an international telecom company and ran follow-up experiments. When customers were already angry, humanlike chatbots lowered satisfaction, and when problems went unresolved, they hurt evaluations of the company and reduced purchase intentions, because the humanlike design raised expectations the bot could not meet. Setting expectations clearly at the start of the chat reduced the damage.
| What customers say about AI in customer service (Gartner) | Share |
|---|---|
| Prefer companies not use AI for customer service | 64% |
| Would consider switching if a company used AI for service | 53% |
| Would try a chatbot again after a bad experience | 27% |
Source: Gartner press releases, July 9, 2024 (survey of 5,728 customers, December 2023) and September 2, 2026 (the 27 percent figure as reported in coverage of the release; survey details were not published in the coverage reviewed).
Finding 5: The Most-Cited Chatbot Statistics Can’t Show Cause and Effect
Search for chatbot conversion statistics and the same figures appear repeatedly: customers who chat are 5 times more likely to complete a purchase (attributed to Tidio), 85 percent more likely to convert (Intercom), and live chat produces a 40 percent increase in conversion rate (ICMI), among others compiled by chat-software vendors. These numbers are usually undated, come from companies that sell chat tools, and compare visitors who chose to start a chat with visitors who did not.
That comparison has a built-in bias. A visitor who opens a chat to ask about availability or pricing is usually already close to buying, and many would have converted with or without the chat. Only randomized experiments, which compare similar visitors with and without access to a chatbot, can isolate the chatbot’s effect. The best of those found a 21.7 percent lift, real and meaningful, but far smaller than 5 times.
| Claim, as commonly cited | Attributed to | What it measures | Shows causation? |
|---|---|---|---|
| Customers who use live chat are 5 times more likely to complete a purchase | Tidio | Chatters vs. non-chatters | No |
| Chatters are 85% more likely to convert into customers | Intercom | Chatters vs. non-chatters | No |
| Live chat leads to a 40% increase in conversion rate | ICMI | Company-reported results | No |
| AI pre-sale chatbot raised conversion 21.7% | Columbia and Zhejiang field experiment (2025) | Randomized: chatbot vs. none | Yes |
Source: Vendor claims as compiled by JivoChat, “52 Live Chat Statistics You Need to Know” (accessed November 2026); original publication years were not stated.
Finding 6: For Local Businesses, Accuracy and Handoff Matter More Than the Widget
The research gap is clear: we found no published randomized study of chatbots on the websites of local service businesses, where most customers convert by phone call, form submission or appointment request rather than online checkout. The closest evidence comes from online retail and financial services, so results for a contractor or clinic may be larger or smaller. What carries over is the pattern: chatbots help most when they answer common questions quickly, are available after hours, serve customers who are new to the business and pass people smoothly to a human.
Accuracy is a business risk as well as a customer-experience issue. In Moffatt v. Air Canada (February 2024), British Columbia’s Civil Resolution Tribunal held the airline liable for negligent misrepresentation after its website chatbot gave a customer wrong information about bereavement fares, rejecting the argument that the chatbot was a separate entity. For a local business, a bot that misquotes a price, promises a same-day appointment that does not exist, or gives health or legal guidance can cost more than it earns.
The Evidence at a Glance
| Study | Setting and design | Sample | Key result |
|---|---|---|---|
| Columbia and Zhejiang field experiments (arXiv, 2025) | Cross-border e-commerce platform; randomized GenAI pre-sale chatbot, 2023 to 2024 | 44,614 shoppers | Conversion +21.7%; sales per shopper +16.3%; +25% with human escalation |
| Luo, Tong, Fang and Qu (Marketing Science, 2019) | Fintech outbound sales calls; randomized AI vs. human agents and disclosure timing | 6,255 customers | Undisclosed AI matched top humans; disclosure before the call cut purchases 79.7% |
| Nayak and Aravindakshan Nair (J. Business Research, 2025) | Chatbot shopping app in India; randomized English vs. English plus Hindi | 12,635 new users | Daily purchases up about 132%; uninstalls also rose |
| Jirpongopas et al. (arXiv, 2025) | U.S. travel-planning site; randomized GPT-4o assistant tone | 282 chat users | Subscriptions about 14 to 15% with tone instructions vs. 5.5% without |
| Crolic, Thomaz, Hadi and Stephen (J. Marketing, 2022) | Telecom chat logs plus experiments | About 35,000 chats | Humanlike bots hurt satisfaction and purchase intent among angry customers |
| Gartner customer surveys (2024, 2026) | Surveys of customers | 5,728 (2024) | 64% prefer no AI in service; 27% would retry a bot after a bad experience |
What this means for local businesses
So do website chatbots increase conversions, and do AI chatbots work for small businesses? The honest answer from the evidence is that they can, under the right conditions, but the proof comes mostly from large online stores, and the way you set the bot up matters more than whether you have one. Here is how we help North Atlanta clients turn that into a plan.
First, treat a chatbot as an experiment, not a sure thing. Run it for 60 to 90 days and compare booked calls, form submissions and appointments with the same period before, or show it to only half of your visitors. Count leads and revenue, not chats. That is also how you measure chatbot ROI honestly, and it mirrors the discipline in our research on the cost of a slow lead response and on what actually drives website ROI.
Second, always offer a human, and make the handoff easy. The strongest results came from bots that could escalate to a person, so show your phone number, offer a callback and route after-hours chats to a follow-up the next morning. A chatbot that captures after-hours leads only pays off if a real person closes them.
Third, limit the bot to what it can answer correctly. Load it with your real hours, service area, pricing ranges and booking rules, and have it say let me connect you instead of guessing, especially on price, availability, health or legal questions. Accuracy is where the legal and reputational risk lives, and it is a core part of our AI consulting work with local businesses.
Fourth, be upfront that it is a bot, and set expectations. Disclosure is increasingly required and customers are skeptical, so a clear, friendly introduction that explains what the assistant can help with reduces frustration. Fifth, serve the customers you actually have: if many of them speak Spanish or another language, a multilingual assistant may matter more than any other feature. Underneath all of it, the website still has to convert, which is why a fast, clear website and strong search visibility do more of the heavy lifting than any widget. You can see more of our work in our original research library, and you can download the full research report as a PDF to share with your team.
About Make It Loud
Make It Loud Digital Marketing is a Metro Atlanta digital marketing agency serving small and midsize businesses across Gwinnett, Forsyth, Hall and Cherokee counties and beyond. The agency provides search engine optimization, generative engine optimization (GEO) and AI search visibility, Google Business Profile management, website design, pay-per-click advertising, social media, podcast production and AI consulting for home service, healthcare, professional service, automotive and nonprofit clients. More research is published at makeitloud.net/research.
Limitations
None of the experiments reviewed involved local service businesses, and several were conducted outside the United States, in different languages and product categories. The e-commerce experiment did not name the platform, and the travel and language studies were small or contain internal inconsistencies noted by reviewers. The 2026 Gartner figure is reported from news coverage of the release rather than a published survey report. Results from 2019 may not reflect current generative AI assistants. Vendor statistics are reported as compiled by a chat-software vendor and were not independently verified. This report is a review and interpretation of outside research, not a Make It Loud experiment or a measure of any client’s results, and it is not legal advice; businesses should review disclosure obligations with counsel.
Sources
Field experiments. “Generative AI and Sales Productivity: Field Experiments in Online Retail,” arXiv:2510.12049 (2025); Luo X., Tong S., Fang Z., Qu Z., Marketing Science 38(6):937 to 947 (2019); Nayak A., Aravindakshan Nair A., Journal of Business Research 190:115158 (2025); Jirpongopas L., et al., arXiv:2509.14259 (2025).
Crolic C., Thomaz F., Hadi R., Stephen A. “Blame the Bot: Anthropomorphism and Anger in Customer-Chatbot Interactions,” Journal of Marketing (2022).
Gartner and JivoChat. Gartner press releases, July 9, 2024, and September 2, 2026; JivoChat, “52 Live Chat Statistics You Need to Know” (vendor compilation, accessed November 2026).
Legal. Moffatt v. Air Canada, 2024 BCCRT 149; California Business and Professions Code sections 17940 to 17943 (B.O.T. Act, SB 1001); Utah Artificial Intelligence Policy Act as amended by SB 226 and SB 332 (2025).
Supports these services
Related topics
Explore the research knowledge graph
How this study connects to our other research, real client outcomes, the problems it helps solve, and practical guides.
Related studies
Problems we solve
About the Researcher

Cliff Tillery
Chief Operating Officer
Principal Researcher
Cliff Tillery is Chief Operating Officer and Principal Researcher at Make It Loud, where he leads the firm's original research initiatives. With a background spanning journalism, business administration, and behavioral healthcare leadership, he brings a research-driven, evidence-first approach to digital marketing, guided by a single principle: diagnose before you prescribe.
Follow our research on Google
Enjoy our research? Add Make It Loud as one of your preferred sources on Google to see more of our future studies.
