bMorgan/SpellingSpider
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

init commit

Browse source
This commit is contained in:
bMorgan01 2022-09-01 08:10:55 -06:00
commit 9d2c4f631b
2 changed files with 129 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

9
crawl.conf Normal file
View file

@ -0,0 +1,9 @@
# Domain Ex: benrmorgan.com
benrmorgan.com
# Prefix Ex: http://www.
http://www.
# Target path Ex /var/www/html/sitemap.xml or ./sitemaps/sitemap.xml
./
# Ignore urls containing Ex: /files/
/files/
/images/