Commit Graph

1373 Commits

Author SHA1 Message Date
Arun Prakash Jana
f9621e4b1a
Reduce redundant newline 2018-08-06 21:18:57 +05:30
Arun Prakash Jana
99b82c3055
Fix #300: detect non-tag keywords 2018-08-06 21:13:02 +05:30
Arun Prakash Jana
c0400c72c3
Merge pull request #301 from rachmadaniHaryono/feature/pytest
fix for new feature
2018-08-06 19:01:48 +05:30
rachmadaniHaryono
f823a6aecc chg: test: simplify test_get_PoolManager 2018-08-06 15:51:36 +08:00
rachmadaniHaryono
9d2b414384 fix: test: test_add_rec_add_invalid_url 2018-08-06 15:47:38 +08:00
rachmadaniHaryono
41b8785a4e fix: test: test_network_handler_with_url 2018-08-06 15:42:46 +08:00
rachmadaniHaryono
551bf5a53d chg: dev: add html5lib package 2018-08-06 15:05:02 +08:00
rachmadaniHaryono
c0af32544a chg: test: use http for example.com 2018-08-06 14:37:28 +08:00
rachmadaniHaryono
3327cf0e7a chg: doc: error on to_temp_file_content 2018-08-06 14:25:22 +08:00
rachmadaniHaryono
8e6c53f404 chg: test: more test on to_temp_file_content func 2018-08-06 14:19:21 +08:00
rachmadaniHaryono
f80e5b241a fix: test: test_to_temp_file_content 2018-08-06 13:58:44 +08:00
rachmadaniHaryono
098d0d5e71 chg: test: remove test_get_page_title 2018-08-06 13:18:13 +08:00
rachmadaniHaryono
77fc10b6ee chg: test: remove BukuHTMLParser 2018-08-06 13:15:55 +08:00
Arun Prakash Jana
7989666250
Do not update title in case of network failures 2018-08-06 07:43:04 +05:30
Arun Prakash Jana
6b58bb9a8d
Add certifi as a requirement. 2018-08-06 06:34:58 +05:30
Arun Prakash Jana
eaea7ddcb0
Use certificate verification.
We keep getting:
"InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised."

Explanation:
https://urllib3.readthedocs.io/en/latest/user-guide.html#ssl
2018-08-06 06:21:26 +05:30
Arun Prakash Jana
bcd2aaf03f
Lint fixes 2018-08-06 05:47:42 +05:30
Arun Prakash Jana
70b6ae318a
Change user agent to X11 mode for full page data 2018-08-05 21:57:20 +05:30
Arun Prakash Jana
62923397b3
More meta tags to look for page description 2018-08-05 21:52:42 +05:30
Arun Prakash Jana
cc6151a775
Handle add and update description from editor 2018-08-05 21:00:01 +05:30
Arun Prakash Jana
5ed1e96368
refreshdb - update desc only is non-empty 2018-08-05 19:13:38 +05:30
Arun Prakash Jana
e0265a5341
Clean newlines in tags and titles 2018-08-05 18:16:03 +05:30
Arun Prakash Jana
5801b8c7e1
Use fetched description and tags from web 2018-08-05 16:37:16 +05:30
Arun Prakash Jana
4b4e5ffe37
Show error once
Show error string only once when deleting a non-existent record by index.

E.g.:

    buku -d non_existent_id

Fix indentation issue.
2018-08-05 09:24:22 +05:30
Arun Prakash Jana
c5b39b8491
A more robust delim_wrap() 2018-08-05 09:12:05 +05:30
Arun Prakash Jana
b463055f92
Fetch title, description and keywords from page 2018-08-05 08:29:19 +05:30
Arun Prakash Jana
5ea4cff4ec
Make bs4 a common import 2018-08-01 19:31:08 +05:30
Arun Prakash Jana
745b6190e4
Check index validity 2018-08-01 19:26:35 +05:30
Arun Prakash Jana
2ad2cbb02b
Merge pull request #298 from rachmadaniHaryono/feature/title
Feature/title
2018-08-01 18:47:08 +05:30
rachmadaniHaryono
b567dd2a64 chg: test: get_page_title func 2018-08-01 20:08:30 +08:00
rachmadaniHaryono
53e1eb2118 Revert "Revert "Revert "Remove redundant message, fix check for 0 records"""
This reverts commit edf5773830.
2018-08-01 19:59:50 +08:00
rachmadaniHaryono
92c6e5c7c7 chg: dev: use bs4 as backup parser 2018-08-01 19:59:33 +08:00
rachmadaniHaryono
1a0b5bd328 new: test: 2 url for testing 2018-08-01 19:57:34 +08:00
rachmadaniHaryono
73c79beca5 chg: dev: use utf8 as default decoder 2018-08-01 19:57:34 +08:00
rachmadaniHaryono
496238262f fix: test: fetch title 2018-08-01 19:57:34 +08:00
rachmadaniHaryono
8180711a33 chg: test: add url for test title fetch 2018-08-01 19:57:34 +08:00
Arun Prakash Jana
79746c1f83
Merge pull request #299 from dkiyatkin/chardet-for-title
Remove double whitespaces from title
2018-08-01 17:26:05 +05:30
Dmitriy Kiyatkin
4983209dfd remove double whitespaces from title 2018-08-01 10:55:12 +00:00
Arun Prakash Jana
f5782a6257
Add refreshdb test 2018-08-01 10:08:14 +05:30
Arun Prakash Jana
60a12a8362
Skip refreshdb test 2018-08-01 10:00:24 +05:30
Arun Prakash Jana
edf5773830 Revert "Revert "Remove redundant message, fix check for 0 records""
This reverts commit 53ee3dc2d7.
2018-08-01 09:59:03 +05:30
Arun Prakash Jana
53ee3dc2d7 Revert "Remove redundant message, fix check for 0 records"
This reverts commit aa29f0cd96.
2018-08-01 09:53:55 +05:30
Arun Prakash Jana
aa29f0cd96
Remove redundant message, fix check for 0 records 2018-08-01 09:52:40 +05:30
Arun Prakash Jana
8a5fe1136b
Fix build break 2018-08-01 09:37:41 +05:30
Arun Prakash Jana
69281444fd Detect page charset from content-type field 2018-08-01 05:54:26 +05:30
Arun Prakash Jana
88c4ccbccb
Merge pull request #295 from rachmadaniHaryono/feature/fix-travis
Feature/fix travis
2018-07-31 18:48:28 +05:30
rachmadaniHaryono
d74190ee92 chg: test: refreshdb 2018-07-31 19:01:18 +08:00
rachmadaniHaryono
b476b40151 fix: text: travis 2018-07-31 15:32:43 +08:00
rachmadaniHaryono
57aec20406 chg: test: add info when test failed 2018-07-31 15:08:19 +08:00
Arun Prakash Jana
883a0ab631
Use UTF-8 to decode HTTP response 2018-07-31 09:36:41 +05:30