cybrneko/searxng-nekosearch
1
0
Fork 0
Code Issues Pull Requests Actions 1 Activity
3,507 Commits 6 Branches 0 Tags
ecf5899153cb58b112fc2c5c745eef91fa36dd6e
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexandre Flament e0142d380e Merge pull request #2211 from return42/fix-centos-7 
Bugfix of:  ERROR: centos-: uWSGI not yet implemented
 2020-09-21 09:06:20 +02:00
Alexandre Flament 4cc8644ec9 Merge pull request #1865 from dalf/uwsgi_static 
[enh] Docker image: uwsgi serves the static files directly.
 2020-09-19 16:04:07 +02:00
Alexandre Flament 375cbd12b3 Merge pull request #2205 from dalf/fix-simple-theme-autocomplete 
[fix] simple theme: fix autocomplete
 2020-09-19 16:02:01 +02:00
Alexandre Flament 530fc4bda7 Merge pull request #2190 from dalf/fix-htmltextextractor 
[fix] searx.utils.HTMLTextExtractor: invalid HTML don't raise an Exception
 2020-09-19 15:59:03 +02:00
Alexandre Flament c23401e9fa [fix] simple theme: fix autocomplete 
AJAX requests send the X-Requested-With HTTP header,
so searx.webapp.autocompleter returns the results with the expected data format.

Related to #2127
Close #2203
 2020-09-19 10:13:05 +02:00
Alexandre Flament 6deb85072a [fix] searx.utils.HTMLTextExtractor: invalid HTML don't raise an Exception 
Close #2188
 2020-09-13 10:28:11 +02:00
Alexandre Flament ae07f4a211 Merge pull request #2185 from dalf/rawtextquery-auto-parse 
[mod] searx.RawTextQuery: the constructor call parse_query
 2020-09-12 15:33:03 +02:00
Alexandre Flament df12ed6e55 [mod] searx.RawTextQuery: the constructor call parse_query 2020-09-12 15:25:58 +02:00
Alexandre Flament bdac99d4f0 Drop Python 2 (5/n): searx.utils.is_valid_lang, input parameter is a str instead of bytes 
Fix bug in translated.py and dictzone.py
 2020-09-10 10:49:42 +02:00
Dalf c225db45c8 Drop Python 2 (4/n): SearchQuery.query is a str instead of bytes 2020-09-10 10:49:42 +02:00
Dalf 7888377743 Drop Python 2 (3/n): objects 2020-09-10 10:39:04 +02:00
Dalf 78df10fb55 Drop Python 2 (2/n): templates 2020-09-10 10:39:04 +02:00
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 2020-09-10 10:39:04 +02:00
Alexandre Flament 272158944b [fix] make docker produces clean tag version (0.17.0 instead of 0.17.0-- for example) 
Fix #2152
 2020-09-10 10:09:15 +02:00
Alexandre Flament 8ffede6140 [fix] docker and k8s: run searx 
fix #2181
 2020-09-10 10:07:49 +02:00
Alexandre Flament d0f9778c2a [mod] results.py: code clean up (#2140) 2020-09-08 10:09:11 +02:00
Alexandre Flament 3397382754 [enh] stop searx when an engine raise an SyntaxError exception (#2177) 
and some other exceptions:
* KeyboardInterrupt
* SystemExit
* RuntimeError
* SystemError
* ImportError: an engine with an unmet dependency will stop everything.
 2020-09-07 15:39:26 +02:00
Alexandre Flament b329058c1a Revert "[enh] test: load each engine to check for syntax errors" 
This reverts commit 4fb3ed2c63.
 2020-08-31 19:00:06 +02:00
Dalf 4fb3ed2c63 [enh] test: load each engine to check for syntax errors 2020-08-28 12:12:32 +02:00
Alexandre Flament ec5adad851 [fix] requirements-dev.txt: set fixed versions for all the dependencies (#2154) 2020-08-27 22:28:51 +02:00
Alexandre Flament 093dd42bb0 Merge pull request #2134 from unixfox/fix/piratebay-engine-date-indentation 
[fix] piratebay engine date and pep8 indentation
 2020-08-11 14:56:19 +02:00
Alexandre Flament f3bfc19a55 Merge pull request #2127 from msafadieh/master 
[fix] fix the browser autocomplete feature.
 2020-08-09 19:21:09 +02:00
Alexandre Flament c3054a10a6 Merge pull request #2121 from gangural/random-sha256-fix 
Fix search error with query 'random sha256'
 2020-08-08 10:45:48 +02:00
Alexandre Flament 6d18769ccf Merge pull request #2101 from mikeri/master 
Update torrentz2 URL from .eu to .is
 2020-07-26 19:33:49 +02:00
Dalf 4a35975b42 [enh] update documentation about docker 2020-07-22 14:58:04 +02:00
Alexandre Flament 61ff5a7aea Merge pull request #2077 from dalf/bangs-no-favicon 
[mod] bangs.json: remove the favicons
 2020-07-15 10:00:12 +02:00
Dalf b339b8b6d6 [mod] bangs.json: remove the favicons 
fix #2076
 2020-07-15 09:15:26 +02:00
Dalf 718715180d [mod] external_bang.py: always read bangs.json with a UTF-8 file. 2020-07-11 12:23:08 +02:00
Alexandre Flament 040adecabf Merge pull request #2026 from return42/fix-yh 
[fix] yahoo engine - changed content_xpath
 2020-06-25 22:34:52 +02:00
Dalf 4c7b787004 [mod] don't try to proxify data URL. 
Previously only image/jpeg was not proxied.
This commit don't proxify all MIME types starting with "image/".

This is a quick fix for the PR #1985 : the google_image engine can returns some data URL.
 2020-06-22 13:57:33 +02:00
Alexandre Flament 2d4d0bdc0c Merge pull request #1947 from jjasonkal/master 
Remove cache of Dockerfile
 2020-05-09 19:25:40 +02:00
Alexandre Flament dc8d975051 Merge branch 'master' into uwsgi_static 2020-04-30 11:50:19 +02:00
Alexandre Flament fbf63f72c7 Merge branch 'master' into requirements-upgrade 2020-04-13 10:34:50 +02:00
Dalf 3973951770 [mod] requirements.txt: upgrade dependency versions. 2020-04-08 15:58:26 +02:00
Dalf a7862ba3c1 [mod] .travis.yml: add Python 3.7 and Python 3.8 2020-04-08 15:42:02 +02:00
Dalf 4dad366146 [fix] travis: fix docker tag. 
the "-dirty" ignores the searx/brand.py and utils/brand.env files.
 2020-04-02 15:06:10 +02:00
Alexandre Flament 2a1db2ac4a Merge branch 'master' into docker-build-fix 2020-03-31 20:14:40 +02:00
Dalf 4bb35e7ef5 [fix] template: fix typo in about.html 2020-03-31 16:52:52 +02:00
Dalf d425e26e80 [fix] fix travis build 2020-03-31 11:59:54 +02:00
Dalf 678d41d75b [enh] Docker image: uwsgi serves the static files directly. 
When the image is built, the static files are compressed with gzip and brotli.
The expires header is set to one day (same as Flask)
There is not etag header (Flask does add an etag header)
 2020-02-25 17:25:11 +01:00
Dalf 88f9ac58f4 [mod] move public instance list to https://searx.space 2020-02-14 13:48:44 +01:00
Dalf 0579d93783 [fix] Disable HTTP keep alive 
Fix HTTP 429 response from filtron
See https://github.com/searx/searx-docker/issues/24
 2020-02-07 19:16:15 +01:00
Dalf 0e7b6c9a03 [mod] infinite scroll: XHR requests are sent with the header "Accept: text/html". 
Fix https://github.com/searx/searx-docker/issues/16
 2019-12-11 14:34:25 +01:00
Dalf f34ac58752 [mod] tidy oscar HTML output 2019-12-10 10:58:30 +01:00
Dalf 85b3723345 [mod] speed optimization 
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
 2019-11-15 09:33:15 +01:00
Alexandre Flament 8e62f7600f Merge pull request #1647 from dalf/no-inline-script 
No inline script for oscar and simple theme
 2019-08-06 12:25:25 +02:00
Dalf 6fbba63c83 oscar and simple themes: no inline script (allow A+ Content-Security-Policy) 2019-08-06 12:14:11 +02:00
Dalf 90b24f1060 manage.sh grunt_build: add node_modules/.bin to the path 2019-08-06 11:49:46 +02:00
Alexandre Flament 12f891da84 Merge pull request #1669 from dalf/engine-fixes 
Engine fixes
 2019-08-05 15:57:33 +02:00
Dalf 1cee2c1796 [fix] bing engine 
before this commit, sometimes there are no results
use a generic user-agent instead of one with the OS "Windows NT 6.3; WOW64"
 2019-08-05 15:46:40 +02:00
Dalf 9a378ca8c6 [fix] etymonline, voat engines 2019-08-05 15:44:22 +02:00
Dalf fcc9587ee9 [fix] fdroid engine 2019-08-05 15:44:02 +02:00
Dalf fbf6b689dd [fix] dictzone engine 2019-08-05 15:43:21 +02:00
Dalf 9ff5001816 [fix] arxiv engine 2019-08-05 15:43:01 +02:00
Alexandre Flament 91a59ff350 [fix] Docker: without -f parameter, settings.yml is not replaced by the new version (#1668) 
Before this commit, the existing settings.yml were always replaced.
 2019-08-03 13:53:14 +02:00
Alexandre Flament 333e54943d [fix] fix monkey patch in test_webapp.py (#1667) 
at the end of test_webapp.py, the monkey patch of searx.search.Search was not revert which lead to side effects on other tests
close #1663
 2019-08-03 13:23:36 +02:00
Alexandre Flament d24e7948eb Merge pull request #1665 from dalf/fix-paging 
[fix] fix paging for the oscar theme after PR #1640
Close #1664
 2019-08-02 20:38:53 +02:00
Dalf ba80a0ba38 [fix] fix paging for the oscar theme after PR #1640 2019-08-02 20:04:37 +02:00
Alexandre Flament 72029d27de [enh] Add timeout limit per request (#1640) 
The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.

In addition, the query can start with <[number] to set the timeout limit.

For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds

For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.

In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).

If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.

Related to #1077
Updated version of PR #1413 from @isj-privacore
 2019-08-02 13:50:51 +02:00
Alexandre Flament 2179079a91 [fix] fix flickr_noapi decoding (#1655) 
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
 2019-08-02 13:37:13 +02:00
Alexandre Flament 02bbbf59d6 [fix] Dockerfile: searx version stores in org.label-schema.version (#1659) 
instead of org.label-schema.schema-version
 2019-08-01 07:51:09 +02:00
Alexandre Flament 1bed39e6cb Merge pull request #1658 from dalf/video-fixes 
Fix dailymotion, google_videos and youtube_noapi engines
 2019-08-01 07:44:30 +02:00
Dalf 0c032c8429 [fix] youtube_noapi engine: fix the title 2019-07-31 08:39:40 +02:00
Dalf 8b7ac56669 [fix] google_videos engine: some results don't a thumbnail 2019-07-31 08:38:45 +02:00
Dalf d44677e226 [fix] dailymotion engine: remove HTML tags from the description 2019-07-31 08:37:51 +02:00
Alexandre Flament 7f56c78876 Merge pull request #1650 from dalf/wikidata 
[fix] wikidata engine: faster processing, remove one HTTP redirection.
 2019-07-29 08:28:50 +02:00
Dalf 6e0285b2db [fix] wikidata engine: faster processing, remove one HTTP redirection. 
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575)
 2019-07-29 07:39:39 +02:00
Alexandre Flament c6ac39bcea Merge pull request #1654 from MarcAbonce/small_fixes 
[fix] Small fixes in Preferences view's text
 2019-07-28 10:31:57 +02:00
Alexandre Flament 4b56a4bfaa Merge pull request #1653 from dalf/bug-fixes 
Various bug fixes
 2019-07-27 09:11:03 +02:00
Dalf 70a01e5f02 [fix] don't merge image results if the img_src is different 2019-07-27 08:52:30 +02:00
Dalf 23611897ec [fix] make sure then engine name is lower case 
Minor fix: "%s engine initialized" display the right engine name
 2019-07-27 08:52:30 +02:00
Dalf 8645ef717e [fix] settings.yml: make sure all engine names are lower case 2019-07-27 08:52:30 +02:00
Dalf 7d8000d965 [fix] "!wp !wp test" send only one request to wikipedia. 
Note that "!general !wp test" is not fixed because the category for "!wp" is "none".
 2019-07-27 08:52:30 +02:00
Dalf 2fab23ab9a [fix] fix the debug message "starting webserver on ip:port" 
was "port:ip"
 2019-07-27 08:52:30 +02:00
Alexandre Flament f34b5cedb1 [fix] fixes google play engines (#1651) 
update commit 87baa74a86
 2019-07-25 09:31:47 +02:00
Alexandre Flament d0dd296424 Merge pull request #1642 from dalf/start-speed 
Speed up start time and decrease CPU load
 2019-07-23 13:18:29 +02:00
Dalf da0ce5880f [fix] fix soundcloud engine, speed up searx start time 2019-07-19 08:59:11 +02:00
Dalf 7e201cbf65 [mod] use cache in _match_language function to speed up searx start time significantly 2019-07-19 08:58:08 +02:00
Alexandre Flament 554a21e1d0 [enh] Add Server-Timing header (#1637) 
Server Timing specification: https://www.w3.org/TR/server-timing/

In the browser Dev Tools, focus on the main request, there are the responses per engine in the Timing tab.
 2019-07-17 10:38:45 +02:00
Alexandre Flament cfcbc3a5c3 [mod] settings.yml: update comments about the morty key. (#1639) 
Fix #1310
Since commit af77ec35d9bd28facdab645a3d57ae340d2b501c Morty accepts base64 encoded key.
 2019-07-17 10:29:28 +02:00
Alexandre Flament 582f3bfcbc Merge pull request #1638 from dalf/suggestions 
Suggestions: use RawTextQuery to make the suggestions URLs.
 2019-07-17 09:20:07 +02:00
Dalf c7220b40ea Suggestions: use RawTextQuery to make the suggestions URLs. Update all themes accordingly. 2019-07-17 09:05:29 +02:00
Alexandre Flament ddee4861ce Merge pull request #1630 from MarcAbonce/bang_fixes 
[fix] Small fixes with bangs in queries
 2019-07-16 11:08:36 +02:00
Alexandre Flament 4d38b8bef7 Merge branch 'master' into bang_fixes 2019-07-16 11:03:52 +02:00
Alexandre Flament f750df8713 Merge pull request #1636 from MarcAbonce/babel_update 
Add Interlingua translation (fix #978)
 2019-07-16 10:48:41 +02:00
Alexandre Flament 2b5a7169a3 Merge pull request #1635 from asciimoo/update-documentation 
README.rst: installation: add searx/searx-docker
 2019-07-13 17:56:17 +02:00
Alexandre Flament efc2a87e77 README.rst: installation: add searx/searx-docker 2019-07-13 17:28:05 +02:00
Alexandre Flament 5b50e116b8 Merge pull request #1629 from dalf/docker 
Update Docker image
 2019-07-13 16:37:39 +02:00
Dalf 45702b77ca embedded iframe (youtube, dailymotion, vimeo): use https 2019-07-13 15:49:59 +02:00
Dalf 609ac5795a .travis.yml: robot tests: remove the xvfb dependency 2019-07-13 15:49:59 +02:00
Dalf fbe40001d3 Update Docker image 
See #1561 , use uwsgi and Alpine Linux

Volume:
/var/log/uwsgi contains error log for 2 days (file uwsgi.log)
/etc/searx contains the settings.yml and uwsgi.ini files.
The docker image creates them if they don't exist.
The two files can be modified after the first run. See below.

Environement variables:
MORTY_URL : external URL of Morty
MORTY_KEY : base64 encoded key
BASE_URL : external URL of Searx
BIND_ADDRESS : internal HTTP port to listen to

Labels : org.label-schema.schema.*

Parameters:
-h : display this help
-d : will update the settings and quit immediately (settings.yml and uwsgi.ini)
-f : always update the settings (previous version saved with suffix .old).
     without this parameter, the new settings are copied with suffix .new

When the Docker image contains newer settings:
- without -f parameter: the new versions are copied to /etc/searx/settings.yml.new and /etc/searx/uwsgi.ini.new.
- with -f parameter:  the old versions are renamed with .old suffix. The new version replaces /etc/searx/settings.yml and /etc/searx/uwsgi.ini

Build using "./manage.sh docker_build", add "push" as parameter also push the Docker image.
The script requires a git repository to work (it makes sure that the last git tag matches searx/version.py)
"git describe" is used to create a meaningful version.
Example : 0.15.0-90-49c5bcb4-dirty (dirty means that the docker image was made with uncommited changes).

Use "docker inspect -f {{.Config.Labels.version}} searx" to get the version of an existing image.

.dockerignore based on .gitignore

.travis.yml: include docker stage
 2019-07-13 15:49:59 +02:00
Alexandre Flament 3b8b862d8c Merge branch 'master' into bang_fixes 2019-07-06 18:48:03 +02:00
Alexandre Flament 3faaf34e25 Fix ./manage.sh robot_tests (#1633) 
doi_resolvers / default_doi_resolver were missing in the settings_robots.yml file, so the test server was not able to start (crash). Since the output wasn't displayed, it was not obvious why the Selenium couldn't connect to searx.
 2019-07-06 18:47:35 +02:00
Alexandre Flament a255651671 Merge pull request #1562 from asciimoo/settings 
Remove some engines : subtitleseeker, seedpeer, swisscows
 2019-05-28 11:14:15 +09:00
Dalf ffe0972f91 Remove some engines : subtitleseeker, seedpeer, swisscows 
http://www.subtitleseeker.com and http://www.seedpeer.eu don't exist anymore.
https://swisscows.ch/ has change : the engine needs to be updated
 2019-05-28 04:06:35 +02:00
Alexandre Flament 6c95ebcff5 Merge pull request #1595 from Venca24/fix_seznam 
[fix] seznam engine
 2019-05-28 07:27:40 +09:00
Alexandre Flament 9318551890 Merge branch 'master' into fix_seznam 2019-05-28 07:23:06 +09:00
Dalf 1418361cbc requirements.txt : update dependencies 2019-05-26 04:31:58 +02:00
Alexandre Flament a84caa22cf Merge pull request #1558 from brunob/patch-1 
simple theme : use instance_name for page title
 2019-04-19 10:04:53 +02:00
Alexandre Flament 9012ad498c Merge branch 'master' into patch-1 2019-04-19 09:59:04 +02:00