cybrneko/searxng-nekosearch
1
0
Fork 0
Code Issues Pull Requests Actions 3 Activity
4,005 Commits 6 Branches 0 Tags
0165e14a7f0400ae10bfb217e4e4bf3992db6bf1
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexandre Flament 0165e14a7f Merge pull request #2632 from searx/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-03-05 11:59:44 +01:00
Alexandre Flament 152f6fc1da Merge pull request #2630 from searx/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-03-05 11:59:20 +01:00
Alexandre Flament d7fb55a774 Merge pull request #2633 from searx/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-03-05 11:59:05 +01:00
Alexandre Flament 10ecc303c9 Fix integration.yml 
Don't run twice the workflow on PR opened from the same repository.
 2021-03-05 11:54:45 +01:00
Alexandre Flament 245f8626dd Fix data-update.yml, run once a month 2021-03-05 11:12:31 +01:00
Alexandre Flament d9a35fc28e Fix data-update.yml 2021-03-05 10:24:54 +01:00
Alexandre Flament 917dff3aab Update data-update.yml 
Allow to manually running the workflow

See:
* https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
* https://docs.github.com/en/actions/reference/events-that-trigger-workflows#workflow_dispatch
 2021-03-05 10:16:59 +01:00
Alexandre Flament b97273df6b Update data-update.yml 
Remove cache
 2021-03-05 09:49:39 +01:00
Alexandre Flament aaae9a209e Merge pull request #2600 from dalf/searx-extra 
Add searx_extra package
 2021-03-05 09:43:39 +01:00
Alexandre Flament b8cd326464 Add searx_extra package 
Split the utils directory into:
* searx_extra contains update scripts, standalone_searx.py
* utils contains the files to build and setup searx.
 2021-03-04 11:59:14 +01:00
Alexandre Flament 1d10ae175c Merge pull request #2618 from thezeroalpha/master 
Fix security vulnerabilities in provided nginx configuration
 2021-03-04 11:27:03 +01:00
Alexandre Flament 6ba37777f7 Merge pull request #2623 from return42/fix-centos7 
[fix] add package `which` to CentOS-7 boilerplate
 2021-03-04 11:19:05 +01:00
Alexandre Flament d26261c5ab Merge pull request #2619 from return42/drop-ubu1604 
[mod] Drop Ubuntu 16.04 (Xenial Xerus) support
 2021-03-04 11:18:21 +01:00
Alexandre Flament 4c2a8aea39 Merge pull request #2620 from return42/fix-git 
[fix] support git versions <v2.22
 2021-03-04 11:17:24 +01:00
Alexandre Flament aac37f288f Merge pull request #2593 from dalf/update-autocomplete 
Update autocomplete
 2021-03-04 10:51:09 +01:00
Alexandre Flament c748fc66cf Merge pull request #2610 from searx/dependabot/pip/master/pylint-2.7.2 
Bump pylint from 2.6.2 to 2.7.2
 2021-03-02 10:12:32 +01:00
Alexandre Flament 87ac6df9fc Merge pull request #2611 from return42/google-scolar 
[enh] google scholar - python implementation of the engine
 2021-03-02 10:05:49 +01:00
Alexandre Flament 63f17d2e4c [enh] autocomplete refactoring, autocomplete on external bangs 2021-03-01 19:12:32 +01:00
Alexandre Flament 4fa1290c11 [fix] answers: don't crash when the query is an empty string 2021-03-01 10:52:39 +01:00
Alexandre Flament e2fb500892 Merge pull request #2608 from return42/unittest2 
[py2to3] use unittest from py3, remove unittest2 from py2
 2021-03-01 10:05:38 +01:00
Alexandre Flament 0c663e25fc Merge pull request #2604 from searx/update_data_firefox_version 
Update searx.data - firefox_version
 2021-03-01 10:03:39 +01:00
Alexandre Flament f77983e174 Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languages 
Fix fetch_languages for Bing
 2021-03-01 09:06:37 +01:00
Alexandre Flament b05f4d0664 Merge pull request #2605 from searx/update_data_currencies 
Update searx.data - currencies
 2021-02-26 12:56:32 +01:00
Alexandre Flament aec5188b51 Merge pull request #2606 from searx/update_data_wikidata_units 
Update searx.data - wikidata_units
 2021-02-26 12:55:51 +01:00
Alexandre Flament 28ed5acde5 Merge pull request #2607 from searx/update_data_ahmia_blacklist 
Update searx.data - ahmia_blacklist
 2021-02-26 12:55:26 +01:00
Alexandre Flament 728e096764 data-update.yml: on PR per file 2021-02-25 08:41:17 +01:00
Alexandre Flament 7c1847d5f2 [mod] add utils/fetch_external_bangs.py 
Based on duckduckgo bangs
Store bangs on a trie to allow autocomplete (not in this commit)
 2021-02-24 18:48:36 +01:00
Alexandre Flament 606aa79e49 Merge pull request #2584 from searx/dependabot/pip/master/sphinx-3.5.1 
Bump sphinx from 3.4.1 to 3.5.1
 2021-02-24 13:52:15 +01:00
Alexandre Flament 5f4a085fc4 Merge pull request #2595 from dalf/update-wikidata-units 
[mod] update wikidata_units.json and fetch_wikidata_units.py
 2021-02-23 17:22:37 +01:00
Alexandre Flament 94c9320c6f Merge pull request #2585 from dalf/update-currencies 
[mod] update currencies.json and fetch_currencies.py
 2021-02-23 17:22:05 +01:00
Alexandre Flament 46ca32c3cc [mod] update currencies.json and fetch_currencies.py 
use a sparql request on wikidata to get the list of currencies.

currencies.json contains the translation for all supported searx languages.

Supersede #993
 2021-02-23 16:42:28 +01:00
Alexandre Flament 93d1da4906 [mod] update wikidata_units.json and fetch_wikidata_units.py 
The fetch_wikidata_units.py result won't change randomly.
See comments in the script.
 2021-02-23 13:10:38 +01:00
Alexandre Flament 9ace31c829 Merge pull request #2588 from datagram1/master 
Added rumble.com video search engine.
 2021-02-21 14:08:02 +01:00
Alexandre Flament 640d6ce778 Update dependabot.yml 2021-02-19 13:08:57 +01:00
Alexandre Flament fd1b8f661b Update dependabot.yml 2021-02-16 18:42:44 +01:00
Alexandre Flament 3ee24b685a Create dependabot.yml 
Only pip packages and ignore pygments, related to https://github.com/searx/searx/pull/2206
 2021-02-16 18:41:34 +01:00
Alexandre Flament 44a6593c13 Merge pull request #2573 from unixfox/yggtorrent 
update yggtorrent url + add it back
 2021-02-16 08:22:07 +01:00
Alexandre Flament 09e28ae779 Merge pull request #2570 from xenrox/improve-peertube 
Improve peertube searching
 2021-02-16 08:17:42 +01:00
Alexandre Flament aef1161633 data-update.yml: run on Friday, 16:05 UTC 2021-02-12 17:19:44 +01:00
Alexandre Flament 45027765e3 Merge pull request #2566 from dalf/remove-yandex 
[remove] yandex engine
 2021-02-12 17:12:07 +01:00
Alexandre Flament 50bde93dbb data-update.yml: run on searx org 2021-02-12 16:56:00 +01:00
Alexandre Flament f59acf4138 data-update.yml: run on Friday 
give the weekend to review the PR
 2021-02-12 11:15:46 +01:00
Alexandre Flament ddab00abf4 Merge pull request #2559 from dalf/upd-duckduckgo 
[fix] duckduckgo engine: "!ddg !g" do not redirect to google
 2021-02-12 11:14:09 +01:00
Alexandre Flament c22d4c764c [fix] duckduckgo engine: "!ddg !g" do not redirect to google 
* searx understand "!ddg !g time" as : send "!g time" to DDG
* !g a DDG bang for Google: DDG return a HTTP redirect to Google

This commit adds a the allows_redirect param not to follow HTTP redirect.

The DDG engine returns a empty result as before without HTTP redirect.
 2021-02-12 11:10:08 +01:00
Alexandre Flament d76660463b Merge pull request #2562 from dalf/mod-json-engine 
[mod] json_engine: add content_html_to_text and title_html_to_text
 2021-02-12 10:58:28 +01:00
Alexandre Flament 7dcf67a47a Merge pull request #2565 from dalf/upd-wikipedia 
[upd] wikipedia engine: return an empty result on query with illegal characters
 2021-02-12 10:57:05 +01:00
Alexandre Flament 2b60d0d243 Merge pull request #2564 from dalf/fix-seznam 
[fix] fix seznam engine
 2021-02-12 10:56:53 +01:00
Alexandre Flament 7e83818879 Merge pull request #2560 from dalf/fix-duckduckgo 
Fix duckduckgo
 2021-02-12 10:56:40 +01:00
Alexandre Flament 63d6ccfbc2 Merge pull request #2557 from dalf/fix-raise_for_httperror 
Fix: activate raise_for_error by default
 2021-02-12 10:56:25 +01:00
Alexandre Flament 74c8b5606f Merge pull request #2541 from return42/mediathekviewweb 
[enh] add engine MediathekViewWeb (API)
 2021-02-11 15:11:26 +01:00
Alexandre Flament 5d9db6c2f7 [remove] yandex engine 2021-02-11 14:28:06 +01:00
Alexandre Flament 35dd069402 [fix] fix seznam engine 
no paging support
 2021-02-11 12:53:19 +01:00
Alexandre Flament 7d6e69e2f9 [upd] wikipedia engine: return an empty result on query with illegal characters 
on some queries (like an IT error message), wikipedia returns an HTTP error 400.
this commit returns an empty result instead of showing an error to the user.
 2021-02-11 12:29:21 +01:00
Alexandre Flament ff84a1af35 [mod] json_engine: add content_html_to_text and title_html_to_text 
Some JSON API returns HTML in either in the HTML or the content.
This commit adds two new parameters to the json_engine:
content_html_to_text and title_html_to_text, False by default.

If True, then the searx.utils.html_to_text removes the HTML tags.

Update crossref, openairedatasets and openairepublications engines
 2021-02-10 16:42:11 +01:00
Alexandre Flament 436d366448 Merge pull request #2544 from mrwormo/congresslibrary 
[Engine] Add Library of Congress engine
 2021-02-10 10:13:46 +01:00
Alexandre Flament eafd27f42a Merge pull request #2556 from dalf/fix-apk-mirror 
[fix] fix apk_mirror engine
 2021-02-10 10:12:37 +01:00
Alexandre Flament c40316d957 Merge pull request #2558 from dalf/remove-google-play-music 
[upd] remove google_play_music engine
 2021-02-10 10:12:21 +01:00
Alexandre Flament d2dac11392 [mod] duckduckgo engine: better support of the language preference 
After the main request, send a second to https://duckduckgo.com/t/sl_h

See https://github.com/searx/searx/issues/2259
 2021-02-09 14:36:43 +01:00
Alexandre Flament 74d56f6cfb [mod] poolrequests: for one (user request, engine) always use the same HTTPAdapter 
The duckduckgo engine requires an additional request after the results have been sent.
This commit makes sure that the second request uses the same HTTPAdapter
= the same IP address, and the same proxy.
 2021-02-09 14:33:36 +01:00
Alexandre Flament 9211cdfe9b [upd] remove google_play_music engine 
Google Play Music has been replaced by Youtube music.
 2021-02-09 11:38:50 +01:00
Alexandre Flament aedf03c0f7 Fix: activate raise_for_error by default 
Fix commit d703119d3a :
Some engines need to parse the HTTP error but
raise_for_error is always set to False in the "request" function.
 2021-02-09 11:27:41 +01:00
Alexandre Flament 5e055b069b [fix) fix apk_mirror engine 2021-02-09 11:02:12 +01:00
Alexandre Flament f03ad0a3c0 Merge pull request #2555 from dalf/fix-github-data-update 
[fix] fix github action data-update.yml
 2021-02-09 10:48:55 +01:00
Alexandre Flament 966a7a1f25 [fix] fix github action data-update.yml 2021-02-09 09:58:59 +01:00
Alexandre Flament e4cc7f13a3 Merge pull request #2542 from kvch/fix-naver-engine 
Fix XPATHs in Naver engine
 2021-02-09 08:52:38 +01:00
Alexandre Flament bec9e30fe7 Merge pull request #2554 from MarcAbonce/zh-variants-in-wikipedia 
Add support for Chinese variants in Wikipedia
 2021-02-09 08:49:59 +01:00
Alexandre Flament 6c513095e4 Merge pull request #2553 from danielhones/improve-results-highlighting-updated 
Ignore double-quotes when highlighting query parts
 2021-02-09 08:39:07 +01:00
Alexandre Flament 1e35c3ccce Merge pull request #2531 from MarcAbonce/fix-browser-locale 
[fix] Get correct locale with country from browser
 2021-02-05 10:55:37 +01:00
Alexandre Flament 321788f14a Merge pull request #2528 from dalf/mod-ci-gh-pages 
[mod] CI: minor changes
 2021-02-04 23:12:27 +01:00
Alexandre Flament 90b9d0d6a8 [mod] CI: minor changes 
* utils/makefile.python: travis-gh-pages renamed ci-gh-pages
 2021-02-02 08:53:57 +01:00
Alexandre Flament 34de715e62 Merge pull request #2500 from dalf/github-action-data 
[enh] every Sunday, call utils/fetch_*.py scripts and create a PR automatically
 2021-02-01 17:16:58 +01:00
Alexandre Flament 1742355eb8 Merge pull request #2499 from dalf/remove-language-support-variable 
[mod] dynamically set language_support variable
 2021-02-01 17:16:18 +01:00
Alexandre Flament ca93a01844 [mod] dynamically set language_support variable 
The language_support variable is set to True by default,
and set to False in only 5 engines.

Except the documentation and the /config URL, this variable is not used.

This commit remove the variable definition in the engines, and
set value according to supported_languages length: False when the length is 0,
True otherwise.

Close #2485
 2021-02-01 17:10:37 +01:00
Alexandre Flament 99244440e4 Merge pull request #2514 from return42/fix-gh-pages 
[fix] Makefile target gh-pages & flatten history of branch gh.pages
 2021-02-01 17:07:08 +01:00
Alexandre Flament 0a8799b834 Merge pull request #2517 from dalf/debug-ci 
Update pyenv pyenvinstall Make targets
 2021-02-01 17:01:34 +01:00
Alexandre Flament d70c5a621a [mod] more robust make pyenv / make pyenvinstall 
"make pyenv" ensures that ./local/py3/bin/python is an executable
 2021-02-01 16:58:04 +01:00
Alexandre Flament 806af50738 Merge pull request #2494 from return42/rm-fabfile 
[fix] remove Fabric file
 2021-02-01 15:09:35 +01:00
Alexandre Flament 71d66979c2 Merge pull request #2482 from return42/fix-google-video 
[fix] revise of the google-Video engine
 2021-01-28 11:11:07 +01:00
Alexandre Flament 0f18e885bf Merge pull request #2479 from Tobi823/master 
Document workaround for using 2 languages simultaneously #1508
 2021-01-27 21:29:42 +01:00
Alexandre Flament b661c3f5d4 Merge pull request #2509 from return42/fix-morty-key 
[doc] improve admin-docs about result proxy (morty) configuration
 2021-01-27 15:31:29 +01:00
Alexandre Flament 6047087aac [mod] utils/fetch_languages.py: write files at the right location 2021-01-24 14:25:27 +01:00
Alexandre Flament 3330cf4a46 [enh] every monday, call utils/fetch_*.py scripts and create a PR automatically 2021-01-24 13:32:39 +01:00
Alexandre Flament f4a17acb7a Merge pull request #2498 from dalf/minor-fix-google-news 
[fix] google_news: avoid one HTTP redirect except for the English results
 2021-01-24 09:13:48 +01:00
Alexandre Flament 96c2996857 Merge pull request #2497 from return42/fix-test.sh 
[fix] lxc.sh - SC2034: ubu2010_boilerplate appears unused.
 2021-01-24 09:06:11 +01:00
Alexandre Flament 8c46b767d0 [fix] google_news: avoid one HTTP redirect except for the English results 
also add
params['soft_max_redirects'] = 1
to avoid false error reporting in /stats/errors
 2021-01-24 08:53:35 +01:00
Alexandre Flament 7d24850d49 Merge pull request #2483 from return42/fix-google-news 
[fix] revise of the google-News engine
 2021-01-23 20:21:09 +01:00
Alexandre Flament 73c86f9bf2 [mod] checker: disable by default 2021-01-19 21:44:48 +01:00
Alexandre Flament 3b7b852aa8 [fix] checker: minor fix about language detection 2021-01-19 21:29:31 +01:00
Alexandre Flament aa887eb375 [mod] checker : replace pycld3 by langdetect 
pycld3 requires the native library cld3
langdetect is a pure python package
 2021-01-19 21:26:04 +01:00
Alexandre Flament 0495e15df4 Merge pull request #2476 from dalf/fix-error-recording-and-checker 
Fix error recording and checker
 2021-01-18 08:29:25 +01:00
Alexandre Flament 67a1aab0d5 [fix] /stats/checker : remove the timestamp field when the checker is disabled 2021-01-18 08:19:53 +01:00
Alexandre Flament d473407ec9 [fix] checker: fix engine statistics 
Without this commit, the URL /stats/errors shows percentage above 100% after the checker has run.
 2021-01-18 08:19:44 +01:00
Alexandre Flament ca76f3119a [fix] error_recorder: record code and lineno about the engine 
since the PR #2225 , code and lineno were sometimes meaningless
see /stats/errors
 2021-01-17 16:25:11 +01:00
Alexandre Flament 80d7411f2c Merge pull request #2452 from kvch/add-wilby-engine 
Add wiby.me engine
 2021-01-16 22:36:31 +01:00
Alexandre Flament b405646749 Merge pull request #2451 from mrwormo/invidious-engine 
[Fix] Invidious Engine
 2021-01-16 19:25:45 +01:00
Alexandre Flament 709dd960f1 Merge pull request #2473 from return42/fix-setup.py 
[fix] setup.py requires pyyaml installed
 2021-01-16 19:05:36 +01:00
Alexandre Flament 1d13ad8452 Merge pull request #2460 from dalf/engine-about 
[enh] engines: add about variable
 2021-01-16 19:05:17 +01:00
Alexandre Flament a4dcfa025c [enh] engines: add about variable 
move meta information from comment to the about variable
so the preferences, the documentation can show these information
 2021-01-14 20:57:17 +01:00
Alexandre Flament 5a511f0d62 [fix] CI: fix docker push 2021-01-14 20:35:10 +01:00
Alexandre Flament 824fe40a28 Merge pull request #2467 from dalf/fix-ci 
[fix] github actions: use ubuntu-20.04 instead of ubuntu-latest
 2021-01-14 17:14:59 +01:00