feat: update robots txt - #1698

Changed pages include:

• index.html - view
• 404.html - view
• authorization.html - view
• changelog.html - view
• data-reporting.html - view
• extending-agent-app.html - view
• extending-chat-widget.html - view
• getting-started.html - view
• guides.html - view
• helpdesk.html - view

There are more changes not included in the list