pjht/buku
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,199 Commits 1 Branch 39 Tags 4.2 MiB
cbdcea7b87
Commit Graph

276 Commits

Author SHA1 Message Date
Arun Prakash Jana
2b90a2319f
Fix race condition, proxy auth handling 
1. In case of a full DB refresh, the threads might enter a race condition while
generating the initial headers. Hence, decoupled the header generation logic.

2. Authorization information in https_proxy is now handled correctly in
urllib3 parse_url() way. This was a miss while completely removing urllib
dependency.

3. Handle exceptions due to malformed proxy URL in multiple places.
 2016-12-11 20:53:48 +05:30
Arun Prakash Jana
6502fd7a64
Support tny.im shortened URL expansion. 2016-12-10 14:05:17 +05:30
Arun Prakash Jana
16069177b3
Remove dependencies atexit and inspect. 2016-12-10 02:23:32 +05:30
Arun Prakash Jana
09bb74b0a4 Use parse_url from urllib3, remove unquote(). 2016-12-10 00:19:14 +05:30
Arun Prakash Jana
6b6315a815 Fix #102.: remove trailing '/'s from search tokens. 
The behaviour is adapted from google's behaviour.
Please see the notes in the bug log for more details.
 2016-12-06 08:42:48 +05:30
Arun Prakash Jana
cf5759d5d7
Update help. 2016-12-03 22:54:14 +05:30
Arun Prakash Jana
9364f3e75d
Reduce params to update_bm(). 2016-12-03 21:02:03 +05:30
Arun Prakash Jana
42ac27bbb1
Support https_proxy with --upstream and --shorten. 2016-12-03 19:33:14 +05:30
Arun Prakash Jana
2becef9b5e
Make number of network threads customizable. 2016-12-03 18:50:53 +05:30
Arun Prakash Jana
f01778fd9b
Set headers and proxy only once in refreshdb() 2016-12-02 00:34:52 +05:30
Arun Prakash Jana
4cbacd58bb
Renew PoolManager for each request. 
This is to fix:
[ERROR] network_handler(), ln 1630: HTTPSConnectionPool(host='www.gnupg.org', port=443): Pool is closed.

Use 1 pool per PoolManager, fresh PoolManager for each request, clear on done.

Q: move on to requests?
 2016-12-01 09:27:15 +05:30
Arun Prakash Jana
3e4f735bb7
check that all records are processed 2016-12-01 08:30:17 +05:30
Arun Prakash Jana
321fc05377
Limit thread count if less than default. 2016-12-01 02:20:36 +05:30
Arun Prakash Jana
5dfa8ace47
Set headers while setting up PoolManager. 
Note: immutable records are not updated from network, so
they don't add to the bookmarks processed per thread count.
 2016-12-01 02:03:15 +05:30
Arun Prakash Jana
4071e76c5c
Use correct format while printing logs 2016-12-01 01:14:23 +05:30
Arun Prakash Jana
f8d4d331d1
Multiple simultaneous notify() calls combine to 1. 
Only one notify() is received in main thread
if multiple threads send call simultaneously.

Test: update only one record. The first 4 notify() come as 1.

Fix: Use a counter to track number of threads completed.
 2016-12-01 00:45:29 +05:30
Arun Prakash Jana
d30dc8109c
5 threads by default in refreshdb(). Write on 32 fetches. 2016-11-30 04:21:11 +05:30
Arun Prakash Jana
d7f0c7a143 Merge pull request #98 from DamianSiniakowicz/master 
draft: multithreaded refreshdb
 2016-11-30 01:49:37 +05:30
Arun Prakash Jana
3331e608cb
Prepare for version 2.7 release. 2016-11-30 01:19:24 +05:30
Damian
147c8d6469 final draft : cleaned everything up 2016-11-28 22:47:30 -05:00
Damian
ae09e6bf99 commented out close_quit on line 2561 2016-11-28 22:22:14 -05:00
Damian
93715929c6 removed multi-connection solution 2016-11-28 20:53:32 -05:00
Damian
1965956dbf Merge remote-tracking branch 'jarun_master/master' 
jarun made a change before I fixed my pull request
 2016-11-28 20:47:18 -05:00
Arun Prakash Jana
46f2f5a1bd Remove redundant DB commit on delete. 2016-11-28 22:21:19 +05:30
Damian
b8cc710461 draft: multithreaded refreshdb 2016-11-27 23:00:42 -05:00
Arun Prakash Jana
c9f8b124b1 Optimize tag delete and replace APIs. 2016-11-28 00:32:06 +05:30
Arun Prakash Jana
6a2471cd3e
Support --noprompt in subprompt. 2016-11-27 20:44:23 +05:30
Arun Prakash Jana
39f307358b Update features and dependencies. 2016-11-27 09:09:57 +05:30
Arun Prakash Jana
f1dd53f1bb
Fix #97: Show index in Json output. 2016-11-27 07:46:54 +05:30
Arun Prakash Jana
314ee1c840 Documentation update. 2016-11-26 20:26:23 +05:30
Arun Prakash Jana
6beed4e183
Update program help. 2016-11-25 23:40:25 +05:30
Arun Prakash Jana
a28bd09013 Shorten debug API calls. 2016-11-23 00:59:26 +05:30
Arun Prakash Jana
de18fd0e0b
Show record when removing tag. 2016-11-21 19:33:04 +05:30
Arun Prakash Jana
b82b2194bf
Show prompt when listing tags. 2016-11-20 23:50:23 +05:30
Arun Prakash Jana
b4fe5c5fd7
Search results honour --format if --json is also specified. 2016-11-20 23:07:04 +05:30
Arun Prakash Jana
3a39685a1b
Update searchdb to handle special keywords. 2016-11-20 22:40:54 +05:30
Arun Prakash Jana
1609b5a135 API to fix tags from earlier versions. Update image. 2016-11-20 20:57:17 +05:30
Arun Prakash Jana
ff8bc645a6 Handle invalid regex search expression. 
Example query:
$ buku --sreg c[m-l]x

Updated image.
 2016-11-20 20:00:59 +05:30
Arun Prakash Jana
c6a0ec7b0d
Show usage count of tags in listing. 2016-11-20 19:01:02 +05:30
Arun Prakash Jana
63bce1004d Store tags in lower case. 2016-11-20 12:46:23 +05:30
Arun Prakash Jana
e56c04a315
Handle exception if no tags in DB. 2016-11-20 11:11:09 +05:30
Arun Prakash Jana
94bd49aa37
Username and password support in proxy auth. 2016-11-18 22:06:09 +05:30
Arun Prakash Jana
1aee1232f7
Support only https_proxy. 2016-11-18 20:04:37 +05:30
Denis A. Altoé Falqueto
f734bf298f Support http proxy environment variables 
If defined, the variables http_proxy, HTTP_PROXY, https_proxy or
HTTPS_PROXY will be used to create a proxied pool manager.

If the prefix https:// is used by the url, it will try to use
the corresponding proxy variables. If not defined, it will
fallback to http_proxy and HTTP_PROXY.

Signed-off-by: Denis A. Altoé Falqueto <denisfalqueto@gmail.com>
 2016-11-17 10:20:10 -02:00
Arun Prakash Jana
d4358e113c Try with http if scheme is missing in URI. 
urllib3 requests can work around this, e.g., google.com is resolved.
However, the browser fails if there's no scheme in the URI.
The observation is based on Google Chrome's behaviour.
So we try http://google.com.

Currently there is no way to override this (i.e. try the URI as it is). We'll
need to work on it if there a bug report on this 'forced' behaviour.
 2016-11-16 21:37:23 +05:30
carlosam
a34fe4bcaf Remove warning when adding HTTPS URL 2016-11-16 00:29:00 +00:00
Arun Prakash Jana
ee3309b154
Use uniform error messages at prompt. 2016-11-15 21:14:26 +05:30
Arun Prakash Jana
e50fb21974 Use a single error string everywhere. 2016-11-15 19:40:28 +05:30
Arun Prakash Jana
de7d1cd463 Open bookmarks by tag list index at prompt 2016-11-15 19:27:16 +05:30
Arun Prakash Jana
31af6ee38e Remove deep search option as a class member. Update help. 2016-11-15 00:12:23 +05:30
Arun Prakash Jana
97c3500138 Add prompt help. Reformat program help. 2016-11-14 23:54:05 +05:30
Arun Prakash Jana
7f75706523
Reverse color at concise prompt. 2016-11-14 00:10:47 +05:30
Arun Prakash Jana
0e936c13a4
Continuous search at prompt. New key q to quit. 2016-11-13 23:22:00 +05:30
Arun Prakash Jana
8441cebd50
Handle regex search independently. 2016-11-13 14:33:19 +05:30
Arun Prakash Jana
32b25e3d4f
Override feed(), remove redundant API to reset parser. 2016-11-13 00:02:38 +05:30
Arun Prakash Jana
299d95aa45 Documentation update. 2016-11-12 23:37:10 +05:30
Arun Prakash Jana
03b351b11e Add URL shortening support. 2016-11-12 21:20:23 +05:30
Arun Prakash Jana
5e0bc15e28
Support custom db file. Optimizations. 2016-11-12 08:08:28 +05:30
Arun Prakash Jana
b387fc95e1
Use user-agent. Simply network_handler(). 2016-11-09 22:44:31 +05:30
Arun Prakash Jana
22bec5871d Modify strings for MIME HEAD fetch 2016-11-09 01:59:02 +05:30
Arun Prakash Jana
b418b95099
Use only one instance of BHTMLParser 2016-11-09 01:13:53 +05:30
Arun Prakash Jana
fd30317265
Make page title a parser member. Rename variables. 2016-11-09 00:22:34 +05:30
Arun Prakash Jana
d521460bdc
top level domains can have 1 to 24 characters. 2016-11-08 23:43:32 +05:30
Arun Prakash Jana
809e6d155a
Use small b in ascii color code 2016-11-08 23:10:16 +05:30
Arun Prakash Jana
ac645e8140
Handle bad URLs, fetch only HEAD for known mimes 
Add API to check URL validity
Add API to detect URLs to some non-page mimes which sans title
network_handler() returns more meaningful tuple
Handle interrupt during multiple indices and/or range update
With urllib3 URLs without preceding http(s) can be handled
 2016-11-08 23:02:45 +05:30
Arun Prakash Jana
c40cd302bf Show only error and skipped if --tacit is used 2016-11-08 12:15:39 +05:30
Arun Prakash Jana
c80a5a8e82 Remove redundant retry count 2016-11-08 11:15:24 +05:30
Arun Prakash Jana
fabdc773b6 Use requests module for upstream version check 
The requests module uses urllib3 internally.
 2016-11-08 10:58:24 +05:30
Arun Prakash Jana
0b819944b8
Remove move_legacy_dbfile() API. We have mergedb(). 2016-11-08 08:45:11 +05:30
Arun Prakash Jana
15892e9eea
Fix markdown import issue with URL having brackets:wq. 2016-11-08 01:56:03 +05:30
Arun Prakash Jana
a7d3ce89a0 Remove metavar param, shorten argparse.SUPPRESS. 2016-11-07 23:28:54 +05:30
Arun Prakash Jana
ed01edf9fb
Remove option --st, redundant 'dest' in addarg. 2016-11-07 23:05:02 +05:30
Arun Prakash Jana
f314a19e8b
Use urllib3 to fetch titles and more... 
Do not fetch title for txt and PDF links.
Disable logging.WARNING if debug mode is not enabled.
Support deflate compression.
 2016-11-07 20:43:08 +05:30
Arun Prakash Jana
92093cfc63 1. Check for None resp before logging. 
2. Update help
3. Show successful update in green in refreshdb()
 2016-11-07 01:20:06 +05:30
Arun Prakash Jana
80413987d7 Merge branch 'setup-py' of https://github.com/asergi/Buku into asergi-setup-py 2016-11-07 00:48:39 +05:30
Alessio Sergi
4cf02b9fa9 Add setup.py for installation as a system tool. 2016-11-06 19:44:44 +01:00