Learn how to create a knowledge collection and integrate them as a chatbot, an MCP server or a Discord Bot
Go to Scrape menu
Enter the URL you want it to scrape and add them to the knowledge collection
It is important to keep the data that it crawls clean for better results. You can remove unnecessary elements from the page like navigations etc. if you know the CSS selectors
Click Scrape to start the scraping process
Wait till it scrapes all the pages it sees in the process
Once the process is done, Go to collection
Add basic settings like name and system prompts from the settings section
Go to Links to see all the links it scraped
Click any link to view more details about the link
It shows all the content it fetched as markdown. This content will be used as knowledge base to answer questions
Go to MCP to use the knowledge base as a MCP server for your customers or community
Go to Embed to add the chatbot into your website. You can find more settings such as the size of the window, sample questions etc.
Go to Integrations to add the CrawlChat Discord Chatbot to your Discord server. It answers the questions and also learns from the conversations.
You can copy the link for the chatbot and use it to test your collection.
This is how the chatbot looks like when you go to the link you copied
It answers the questions from the knowledge collection and also shows the resource links!
You successfully completed the guide!