a PHP spider and search engine
Sphider 4.1.0-MB, SphiderLite 2.1.0 have been released
18 April 2022 - This update removes the re-index restart capability (This is why.) and includes improved indexing using sitemaps.
To upgrade, simply replace the files noted in the changelog, then run /admin/version_update.php.
Sphider 4.0.2-MB, SphiderLite 2.0.2 have been released
27 February 2022 - This is a minor update. Testing revealed a couple PHP 8 vulnerbilities. These have been corrected.
To upgrade, simply replace and add the files noted in the changelog, then run /admin/version_update.php.
16 December 2021 - The most recent releases (Sphider 4.0.1-MB and SphiderLite 2.0.1) have been running under PHP 8.1 for two weeks on a test machine. No errors or warnings so far. Testing will continue, but it looks safe under PHP 8.1.
Sphider 4.0.1-MB, SphiderLite 2.0.1 have been released
28 July 2021 - This is a minor update. Russian stemming has been added. The are a few miscellaneous updates and corrections.
Sphider 4.0.0-MB, SphiderLite 2.0.0 have been released
18 January 2021 - The backup and restore utilities have been reworked to use MySQL directly. This provides higher dependability than depending on PHP. Also, a limited ability to resume a re-index process which has been interrupted has been introduced. The process to determine page character set has been enhanced. Language file conversion to Unicode has been completed. Obsolete versions of code have been removed and general code cleanup done. Further safeguards against indexing of illegal characters has been implemented. SphiderLite has had more remnants of the full version removed.
SphiderLite 1.3.1 has been released
31 December 2020 - A critical flaw was discovered in the SphiderLite indexing code. This has been corrected in this version. ALL USERS OF PREVIOUS VERSIONS OF SPHIDERLITE ARE URGED TO UPGRADE TO 1.3.1! In addition to fixing this CRITICAL FLAW, SphiderLite has also improved the determination of a page character set, has improved filtering while indexing keywords, and improved removal of emojis.
Sphider 3.6.0-MB and SphiderLite 1.3.0 have been released
26 December 2020 - A potential runaway regular expression resulting in missing titles has been corrected. Crawl performance has been improved by fixing a bug that caused Sphider to try to crawl pages returning codes like 301, 401, 403, and 404. The absence of a robots.txt file on sites being crawled was generating warning errors, and this has been corrected. More potential PHP 8 errors have been averted. More obsolete code has been removed. The MB version now reports when a feed becomes invalid.
SphiderLite 1.2.3 has been released
20 December 2020 - A number of PHP warnings were removed, all of which could have been fatal in PHP 8. Also a minor improvement was made when parsing title tags. This is the companion to yesterday's Sphider-3.5.3-MB release.
Sphider 3.5.3-MB has been released
19 December 2020 - More deprecated code was removed and a number of PHP warnings, all of which could have been fatal in PHP 8. Also a minor improvement was made when parsing title tags. A similar change will be released for SphiderLite very soon.
Sphider 3.5.2-MB and SphiderLite 1.2.2 have been released
9 December 2020 - It was found that when a search returned results, if a new search was performed which should have found nothing, the previous search was returned! This was corrected.
Sphider 3.5.1-MB and SphiderLite 1.2.1 have been released
8 December 2020 - A correction was made to a unicode function that will allow faster searches for installations with the mbstring extension installed.
Sphider 3.5.0-MB and SphiderLite 1.2.0 have been released
6 December 2020 - The text search function was split to improve efficiency. Previously, the search was repeated for each page of results displayed. Now the search is performed only once, and the appropriate subset displayed for each page viewed.
Sphider 3.4.5-MB and SphiderLite 1.1.5 have been released
30 November 2020 - Deprecated code which would have caused an error in PHP 8 was removed. A parsing issue with the robots.txt file was resolved. Two options were removed from the database tab. The Optimize function has been inert since Sphider changed to the Innodb database structure, and the Truncate function proved to be risky due to the addition of foreign key constraints.
Sphider 3.4.4-MB and SphiderLite 1.1.4 have been released
7 November 2020 - Fixed a critical error in the truncate tables routine.
Sphider 3.4.3-MB, SphiderLite 1.1.3, and Sphider 2.4.3-PDO has been released
30 August 2020 - These are a minor releases. Corrected several issues due to PHP 7.3 and 7.4. The PDO release is a one time maintainence release to keep it functioning.
Sphider 3.4.2-MB and SphiderLite 1.1.2 has been released
15 August 2020 - This is a minor release. Some code in stem_class.php had become deprecated in PHP 7.4 and was updated.
Sphider 3.4.1-MB and SphiderLite 1.1.1 has been released
18 July 2020 - This is a minor release. The only change is that the version of jquery used in searching has been updated to 3.5.1.
Typo corrected in tables.sql
5 July 2020 - A typo was discovered in the tables.sql script for both Sphider aqnd SphiderLite. This has been corrected and any future downloads will reflect the correction. If you previously downloaded Sphider 3.4.0-MB or SphiderLite 1.1.0 and install using the SQL script, you may encounter an error. The correction follows:
Line 239 (237 in SphiderLite) in tables.sql is:
ENGINE=InnoDB DEFAULT CHARSET=utf8mb4COLLATE=utf8mb4_general_ci;
It should be:
ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
Sphider 3.4.0-MB and SphiderLite 1.1.0 has been released
7 December 2019 - Both of these have a couple minor bug fixes and a new feature. It is now possible to create a sitemap from any previously indexed site.
SphiderLite 1.0.0 has been released
7 November 2019 - SphiderLite has the same site indexing capabilities as the regular version, but has had image and RSS functions removed.
Sphider 3.3.0-MB has been released
7 September 2019 - The database was altered to provide foreign key restraints. The admin functions were simplified to take advantage of database control provided by the added restraints. Sphider functionality is identical to the prior version.
Sphider 3.2.1-MB has been released
3 August 2019 - A fix was made to correct index omissions of words containing certain compound characters.
Sphider 3.2.0-MB has been released
16 June 2019 - A fix was made to correct missing keyword-link relationships. New features are the ability to set a minimum score for search results and the ability to use a 5 star system to rank the relevancy of results. Iframe support has been added.
Sphider 3.1.1-MB and 2.4.2-PDO has been released
16 May 2019 - A fix was made to correct indexing of words composed of non-English characters.
Sphider 3.1.0-MB and 2.4.2-PDO have been released
15 May 2019 - Sphider 3.1.0-MB replaces both Sphider 2.4.1 and 3.0.0-MB. Like 3.0.0-MB, 3.1.0-MB is multibyte string capabable, but without the requirement for the PHP mbstring extension. If mbstring is present, it will be used. If not, it will be emulated. Several character set encoding issues have been resolved, word recognition improved, more reliable meta description indexing, and presentational improvements have been made. Sphider 2.4.2-PDO has eliminated the possibility of certain UTF-8 characters being misinterpreted as ISO-8859-1. As previously announced, the PDO branch will remain available and critical fixes will be made as needed, but no future product enhancements are planned.
Versions 2.4.1, 2.4.1-PDO, and 3.0.0-MB have been released
29 April 2019 - Sphider 2.4.1 and 2.4.1-PDO are maintenance releases. Primary code changes involve how SQL errors are reported in the event a statement preparation fails. Preparation errors should never occur, but better to have good information in the off-chance it does happen. Also, a coding error introduced in 2.4.0 that prevented indexing from a browser in some instances has been corrected.
Sphider 3.0.0-MB is actually 2.4.1 with one BIG change... String handling is now equipped to handle multi-byte character strings. Since the PHP mbstring extension is used, and some hosts do not always provide this extension, Sphider 3.0.0-MB may not run for everyone. Also, being the beginning of a new approach to indexing and searching, some people may feel more comfortable sticking with what they KNOW will work! We feel this will ultimately prove to be the better way.
Version 2.4.0 Released, Legacy and PDO
10 April 2019 - Sphider 2.4.0 has completely re-worked the settings table, giving a number of new search options. Word stemming has been updated and now works with a number of languages other than English. A number of bugs have been corrected. The selection of search templates has been changed with elimination and replacement for the majority.
Downloads of SQLite and PostgreSQL to end
16 March 2019 - The download links for the SQLite and PostgreSQL editions of Sphider will be removed in April. Forum support for these editions will continue.
Version 2.3.1 Released, Legacy and PDO
5 March 2019 - Sphider 2.3.1 has corrected the issue with indexing and searching for words composed of non-Western characters.
Version 2.3.0 Distribution on Hold
4 March 2019 - Sphider 2.3.0 is being re-evaluated due to some unresolved issues with searches in foreign languages. The 2.2.0 versions are still available in the interim.
Version 2.3.0 Released
25 Feb 2019 - Sphider 2.3.0 and 2.3.0-PDO have been released. Consult the change logs to see what's new in this release. The Sphider User Guide is no longer included in the download packages, but is still available as a separate download. The User Guide has also been updated to reflect the changes.
Updates discontinued for SQLite and PostgreSQL
1 December 2018 - The decision was made not to produce any new updates for the SQLite and PostgreSQL forks of Sphider. They are frozen at release 2.1.0. There just hasn't been sufficient demand to make the time and effort required to maintain these "novelties" worthwhile. User support will still be provided via the forum.