Enrich your chatbot's knowledge with all the information from your website? You can do that at the press of a few buttons with the Web Scraper!
In addition to incorporating your own domain knowledge, you may want to include even more information in your chatbot. With the help of the Web Scraper, all relevant data from specific websites is "scraped" and integrated into your chatbot's knowledge base! This means you no longer have to manually input all the data, making chatbot maintenance much simpler. In this article, we'll explain how to use the Web Scraper effectively.
In this article, you'll learn more about what exactly a Web Scraper is and how it works (technically).
1. Accessing the Web Scraper
To get started, create a new chatbot or open an existing one where you want to include information from a specific website. Go to the "Context manager" menu to find the Web Scraper.
2. Adding the website
Once in the Web Scraper environment, you can easily add and manage your website. Enter the URL of the website from which you want to extract information and click on 'Start Scraping'.
The status of the scraping process and the last time the website was scraped will be displayed. For larger websites, it may take some time to complete the entire scraping process.
3. Scrape again
When the content on the website changes, you can click on 'Scrape' again to re-scrape the website.
4. Removing a scraped URL
If you want to remove a specific URL from the Web Scraper, simply click on the designated button next to the URL.
Note: Removing a URL will also erase all the knowledge that the chatbot gained from that particular website.
5. Testing the chatbot with website knowledge
Once the scraping process is complete, you can test your chatbot using the knowledge acquired from the scraped website. This can be done through the interactive tester.
Important: If the information on your website contradicts the manually added prompts, the chatbot will always prioritize the instructions from the prompts and use that information to generate answers, regardless of the scraped data.