Sphider
a PHP spider and search engine
Each variation of Sphider has a variation in changes made. Updates to the multi-byte edition of Sphider appears by default. You may also view the Lite or (Obsolete) PDO editions.
PDO Sphider
NOT COMPATIBLE WITH PHP 8
Will produce warning messages with PHP 7.4
PHP 7.3 and under recommended
Sphider 2.4.3-PDO, release date 30 August 2020 Update to remove errors when running under PHP 7.3, PHP 7.4 (RAP) Changed file: admin/db_backup.php admin/install.php admin/update_rollup.php include/searchfuncs.php include/simple_html_dom.php sql/tables.sql search.php Sphider 2.4.2-PDO, release date 13 May 2019 Corrected character encoding of several Sphider modules (RAP) Eliminated possiblitiy of utf-8 text being misidenfitied as Windows-1252 or Latin-1 (RAP) Changed files: admin/db_backup.php admin/install.php admin/spider.php admin/spiderfuncs.php admin/update_rollup.php common_template/search_results.php include/commonfuncs.php include/stem_class.php languages/am-language.php languages/cs-language.php languages/da-language.php languages/de-language.php languages/es-language.php languages/et-language.php languages/fi-language.php languages/fr-language.php languages/hu-language.php languages/nl-language.php languages/no-language.php languages/pt-language.php languages/sk-language.php languages/sq-language.php languages/sv-language.php languages/tr-language.php sql/tables.sql Sphider 2.4.1, release date 29 April 2019 Improved emoji removal by ensuring the functiopn is working with utf-8 text (RAP) Added HTTP request headers specifiying utf-8 character set in both admin and search pages (RAP) Fixed issues with SQL error reporting (RAP) Changed files: admin/admin.php admin/configset.php admin/db_backup.php admin/db_main.php admin/install.php admin/rss_spider.php admin/spider.php admin/spiderfuncs.php admin/update_rollup.php common_template/header.php common_template/img_search_form.php common_template/rss_search_form.php common_template/search_form.php include/commonfuncs.php include/searchfuncs.php js_suggest/suggest.php settings/conf.php sql/tables.sql search.php Sphider 2.4.0-PDO, release date 10 April 2019 Reworked the settings table to make maintaince easier (RAP) Added the ability to hide the result number in search results (RAP) Added the ability to show the index date in search results (RAP) Added the ability to hide the url at the bottom of a result (RAP) Added hide/show advanced search to the RSS and image searches (RAP) Dropped option to show previews on the text results page (RAP) Fixed broken "Show only the 2 most relevent links" (RAP) Miscellaneous bug fixes (RAP) Fixed broken image search (RAP) Updated English word stemming to Porter 2 (RAP) Added word stemming capability for 10 more languages (RAP) Corrected some display anomolies for image search results (RAP) Corrected typo in several css files (RAP) Added new css class (RAP) Added new search templates, removed others from distrubution (RAP) Changed files: admin/configset.php admin/db_backup.php admin/install.php admin/spider.php admin/spiderfuncs.php admin/update_rollup.php common_template/img_search_form.php common_template/img_search_results.php common_template/rss_search_form.php common_template/rss_search_results.php common_template/search_results.php include/commonfuncs.php include/searchfuncs.php settings/config.php sql/tables.sql templates/dark/m_search.css templates/dark/search.css templates/nebula/m_search.css templates/nebula/search.css templates/standard/m_search.css templates/standard/search.css search.php New files: include/stem_class.php templates/black/m_search.css templates/black/search.css templates/green/leafy.png templates/green/m_search.css templates/green/search.css templates/grey/m_search.css templates/grey/search.css templates/newspaper/m_search.css templates/newspaper/search.css templates/simple/m_search.css templates/simple/search.css templates/terminal/m_search.css templates/terminal/search.css templates/yellow/logo.png templates/yellow/m_search.css templates/yellow/search.css Deleted files: include/pstem_class.php Removed templates: air, cars, earth, fire, space, sports, water Sphider 2.3.1-PDO, release date 5 March 2019 Corrected problem with spidering and indexing sites using words with non-European characters (RAP) Changed files: admin/db_backup.php admin/install.php admin/spiderfuncs.php admin/update_rollup.php include/commonfuncs.php include/searchfuncs.php sql/tables.sql Sphider 2.3.0-PDO, release date 25 February 2019 Made all templates mobile browser friendly (RAP) Made choice of search forms displayed user configerable (RAP) New stock templates added (RAP) Moved RSS feed maintenance items to their own tab (RAP) Continued to update code to remove tables and deprecated html (RAP) Added the ability to set search page language by url parameter (RAP) Tightened security against $_POST attacks (RAP) Added hard logout to prevent session restart (security) (RAP) Added ability to set minimum image size for indexing (RAP) Replaced each() function, deprecated in PHP 7.2 (RAP) Made image and RSS wildcard searches less obtuse (RAP) Fixed RSS 'or' search (RAP) Reversed logic concerning images and robots.txt (RAP) When adding sites, if url redirects http->https, update url (RAP) Admin link in feed spider results now leads to show feeds tab (RAP) Upgrade all tables to use 4-byte encoding instead of 3-byte (RAP) Changed files: admin/admin.css admin/admin.php admin/auth.php admin/configset.php admin/db_backup.php admin/db_main.php admin/install.php admin/messages.php admin/rss_spider.php admin/spider.php admin/spiderfuncs.php admin/update_rollup.php common_template/footer.php common_template/img_search_results.php common_template/search_results.php include/commonfuncs.php include/detectmobilebrowser.php include/searchfuncs.php settings/conf.php sql/tables.sql templates/*/search.css (x6)(in templates/air, dark, earth, fire, standard, water) search.php install.txt New files: admin/index.html common_template/header.php templates/*/m_search.css (x6)(in templates/air, dark, earth, fire, standard, water) templates/cars/bgimage.png templates/cars/m_search.css templates/cars/search.css templates/nebula/bgimage.png templates/nebula/m_search.css templates/nebula/search.css templates/space/bgimage.png templates/space/m_search.css templates/space/search.css templates/sports/bgimage.png templates/sports/m_search.css templates/sports/search.css Deleted files: templates/*/header.html (x6)(in templates/air, dark, earth. fire, standard, water) templates/mobile (entire folder) Sphider 2.2.0-PDO, release date 3 December 2018 Removed the use of tables from the search forms (RAP) Added css to replace the use of tables (RAP) Added a new template for use on mobile devices (RAP, aided by ReddWebDev) Added a mobile browser auto-detect to the search function (RAP) Cleaned up several language files (RAP) Security fix: Turned off PHP error reporting used for development (RAP)] Increased db user_agent column to 50 (from 15) (RAP, suggested by ReddWebDev) Add a tabbed search toggle to the settings page (RAP) Changed files: admin/admin.php admin/auth.php admin/configset.php admin/db_backup.php admin/db_main.php admin/install.php admin/messages.php admin/rss_spider.php admin/spider.php admin/update_rollup.php common_template/footer.php common_template/img_search_form.php common_template/img_search_results.php common_template/rss_search_form.php common_template/rss_search_results.php common_template/search_form.php common_template/search_results.php include/searchfuncs.php js_suggest/suggest.php settings/conf.php sql/tables.php language/am-language.php language/ar-language.php language/da-language.php language/en-language.php language/fa-language.php language/no-language.php language/sk-language.php language/sq-language.php language/tr-language.php templates/*/search.css (x6)(in templates/air, dark, earth, fire, standard, water) search.php New files: include/detectmobilebrowser.php templates/mobile/header.html templates/mobile/search.css Deleted files: search-161.php Sphider 2.1.0-PDO, release date 23 November 2018 Found wildcard search was broken, fixed it (RAP) Found the negate search broken, fixed it (RAP) Changed files: admin/db_backup.php admin/install.php admin/update_rollup.php include/commonfuncs.php sql/tables.php Sphider 2.0.2-PDO, release date 15 November 2018 Code cleanup (RAP) Purged ^M characters (RAP) Resolved a few issues caused by the release of 2.0.1! (RAP) Updated to later version of jquery, 3.3.1 (RAP) Changed files: admin/db_backup.php admin/install.php admin/spider.php admin/update_rollup.php sql/tables.sql templates/*/header.html (x6)(in templates/air, dark, earth, fire, standard, water) search.php search-161.php Sphider 2.0.1-PDO, release date 13 November 2018 Fixed coding errors in error handling routines (RAP) Changed files: admin/rss_spider.php admin/spider.php admin/spiderfuncs.php search.php Sphider 2.0.0-PDO, release date 26 October 2017 Changed some potentially problematic code (RAP) Improved indexing reliability for pages encoded in other than latin-1 or utf-8 (RAP) Fixed a problem with Reindex All (RAP) Improved responsiveness of "phrase" searches (RAP) Avoided potential problem in "and" and "or" searches (RAP) Fixed a PHP7.1 issue with "too few arguments" for a function (RAP) Added indexing of RSS feeds and associated search (RAP) Integrated Image Indexing into the main spider (RAP) Uncluttered the main search screen when multiple domains are present (RAP) Integrated main search, RSS search, and image search to a single tabbed page (RAP) (Retained search.php from Sphider 1.6.0 as search-161.php for those who desire only a simple content search - slightly modified to stay compatible with other 2.0.0 code changes. To be useful, it needs to be renamed to search.php with the distributed search.php renamed to NEWsearch.php.) Renamed some language files to conform with ISO standards (RAP) (ISO renames): alb- to sq-, cns- to zh_cn-, cnt- to zh_tw-, cz- to cs-, ee- to et-, si- to sl-, swa- to sw-, se- to sv- Created Danish and Norwegian language files (RAP) Updated code to come closer to "best PHP coding practices" (PSR-2) (RAP) (This entailed renaming many of the functions to meet standards rcommendations.) Removed several unused functions (RAP) Updated to later version of jquery, 2.2.4 (RAP) Changed files admin/admin.css admin/admin.php admin/auth.php admin/configset.php admin/db_backup.php admin/db_main.php admin/dbmain.js admin/install.php admin/messages.php admin/spider.php admin/spiderfuncs.php admin/update_rollup.php common_template/categories.php common_template/footer.php common_template/img_search_form.php (new if it doesn't exist) common_template/img_search_results.php (new if it doesn't exist) common_template/search_form.php common_template/search_results.php include/categoryfuncs.php include/commonfuncs.php include/pstem_class.php include/searchfuncs.php js_suggest/suggest.php settings/conf.php settings/database.php languages/*-language.php (all changed) sql/tables.sql templates/*/header.html (x6)(in templates/air, dark, earth, fire, standard, water) templates/*/search.css (x6)(in templates/air, dark, earth, fire, standard, water) search.php New files admin/rss_spider.php calendar/* (new folder, sub-folders and files) common_template/rss_search_form.php common_template/rss_search_results.php language/da-language.php (Danish) language/no-language.php (Norwegian) Deleted files images/images.php (if exists) images/imgsearchfuncs.php (if exists) img_search.php (if exists) include/localutils.php SII_changelog (if exists) SII_Installation_Notes.txt (if exists) Moved files simple_html_dom.php (/images to /include)(new if not exist) Folders /tmp (new) /images (removed if exists) Sphider 1.6.1-PDO, release date 10 August 2017 Fixed a problem which could result in duplicate keywords (RAP) Changed files: db_backiup.php install.php spiderfuncs.php tables.sql update_rollup.php Sphider 1.6.0-PDO, release date 14 July 2017 Added the ability to truncate tables (RAP) Added the ability to clear site data while retaining the site and its settings (RAP) Added the ability of crawling from a sitemap (RAP) Added the option of previewing pages on the results listing (RAP) Added support for the optional Sphider Image Indexer (RAP) Fixed problem that caused suspended indexing not to resume (RAP) Fixed error on sites index screen (RAP) Fixed error in a sphider sub-function (RAP) Fixed another potential problem with table prefixes containing a hyphen (RAP) Removed more deprecated html for HTML5 compliance (RAP) Made coding changes to make porting between database types easier (RAP) Changed files admin.php admin.css commonfuncs.php conf.php configset.php db_backup.php db_main.php install.php install.txt messages.php search.css (x6)(in air, dark, earth, fire, standard, water) search.php searchfuncs.php search_results.php spider.php spiderfuncs.php tables.sql update_rollup.php SphiderUserGuide.pdf Sphider 1.5.4-PDO, release date 29 May 2017 Added the ability to index decimal numbers (RAP) Added a filter to strip emoticons from text (presence of emoticons interferes with indexing) (RAP) Fixed another potential problem with table prefixes containing a hyphen (RAP) Changed many language files to display proberly in UTF-8 (not a linguist, so ...) (RAP) Added Albanian language (Fatih Ibrahimi) Added Amharic language (sheshu) Added Swahili language (mdoja) Changed files admin.php conf.php configset.php db_backup.php install.php searchfuncs.php sphider.php sphiderfuncs.php tables.sql update_rollup.php SphiderUserGuide.pdf Changed files in the languages folder ar-language.php bg-language.php cns-language.php cnt-language.php cz-language.php de-language.php ee-language.php en-language.php es-language.php fa-language.php fi-language.php fr-language.php hr-language.php hu-language.php it-language.php lv-language.php nl-language.php pl-language.php pt-language.php ro-language.php ru-language.php se-language.php si-language.php sr-language.php sk-language.php tr-language.php New files /languages/alb-language.php /languages/am-languarge.php /languages/swa-language.php Deleted cn-language.php; Unused and superceded by cns-language.php and cnt-language.php Sphider 1.5.3-PDO, release date 15 Apr 2017 Fixed a problem where Clean Domains routine would hang when the sites table was empty (RAP) Fixed a problem in which the robots.txt may not be properly recognized and parsed (RAP) Added ability to read the robots.txt file on HTTPS sites (RAP) Fixed a potential PHP error on a break statement (RAP) Fixed a potential problem with table prefixes containing a hyphen (RAP) Changed files: spiderfuncs.php categoryfuncs.php searchfuncs.php search.php admin.php spider.php db_backup.php install.php update_rollup.php tables.sql SphiderUserGuide.pdf Sphider 1.5.2.1, release date 14-Dec-2016 Fixed issue that sphider would abort a crawl when an improperly coded character was encountered on a web page. A non-fatal error is now thrown, but execution contines on the next page. (RAP) Corrected possible database error when updating settings due to a possible null value (RAP) Corrected bug that prevented descriptions from appearing in search results if the settings page was updated (RAP) Sphider 1.5.1.1, release date 08-Feb-2016 ** FUNCTIONALLY EQUIVALENT TO SPHIDER 1.5.1 ** TARGET: SYSTEMS LACKING MYSQLND SUPPORT ** INTENDED AS AN ALTERNATIVE AND NOT AN UPGRADE OR REPLACEMENT Converted SQL from the MySQLi extension to PDO (RAP)