Sped0n 88da65ef67 feat(robots.txt): add robots.txt file to block unwanted bots and allow all other bots to crawl the site
The robots.txt file is added to the layouts directory. This file includes rules to block specific bots from crawling the site. The following bots are blocked: MJ12bot, AhrefsBot, BLEXBot, SISTRIX Crawler, sistrix, 007ac9, 007ac9 Crawler, UptimeRobot/2.0, Ezooms Robot, Perl LWP, netEstate NE Crawler (+http://www.website-datenbank.de/), WiseGuys Robot, Turnitin Robot, Heritrix, pricepi, SurdotlyBot, and ZoominfoBot. All other bots are allowed to crawl the site. The file also includes a sitemap directive to point to the sitemap.xml file.
2023-11-03 10:11:02 +08:00
2023-03-09 08:50:16 +08:00
2023-03-09 09:06:20 +08:00
2023-03-09 08:50:16 +08:00
2023-10-29 12:52:44 +08:00
2023-03-09 08:50:16 +08:00
Description
Bridget is a minimal Hugo theme for photographers/visual artists, powered by SolidJS.
MIT 53 MiB
Languages
TypeScript 63.5%
HTML 18.5%
SCSS 12.5%
JavaScript 3%
Shell 1.5%
Other 1%