Merge remote-tracking branch 'origin/master' into tests

CHANGELOG

@@ -1,3 +1,17 @@
+Buku v2.2
+2016-06-12
+
+Modifications
+- Export bookmarks to Firefox bookmarks formatted HTML
+- Merge Buku database
+- .deb package for Debian and Ubuntu family
+- Switch from PyCrypto to cryptography (thanks @asergi)
+- Append tags support
+- Filter tags for duplicates and sort alphabetically
+- Travis CI integration, more test cases (thanks @poikjhn)
+- Show DB index in bold in search results
+- Several performance optimizations
+
 -------------------------------------------------------------------------------
 
 Buku v2.1

README.md

@@ -28,7 +28,7 @@ Copyright (C) 2015-2016 [Arun Prakash Jana](mailto:engineerarun@gmail.com).
 # Features
 
 - Add, tag, comment on, search, update, remove bookmarks
-- Merge-able portable database, to sync between systems (not released yet)
+- Merge-able portable database, to sync between systems
 - Import/export bookmarks HTML (Firefox, Google Chrome, IE compatible)
 - Fetch page title from web (default), refresh all titles in a go
 - Open search results directly in browser
@@ -46,6 +46,7 @@ Copyright (C) 2015-2016 [Arun Prakash Jana](mailto:engineerarun@gmail.com).
   - [Running as a standalone utility](#running-as-a-standalone-utility)
   - [Shell completion](#shell-completion)
   - [Installing with a package manager](#installing-with-a-package-manager)
+  - [Debian package](#debian-package)
 - [Usage](#usage)
   - [Cmdline options](#cmdline-options)
   - [Operational notes](#operational-notes)
@@ -58,9 +59,9 @@ Copyright (C) 2015-2016 [Arun Prakash Jana](mailto:engineerarun@gmail.com).
 
 ## Dependencies
 
-`buku` requires Python 3.x to work.
+`buku` requires Python 3.3 or later.
 
-Optional dependencies:
+Package dependencies:
 - Encryption: cryptography
 - Import bookmarks: Beautiful Soup
 
@@ -104,17 +105,21 @@ Shell completion scripts for Bash, Fish and Zsh can be found in respective subdi
  - Void Linux repos ( `$ sudo xbps-install -S buku` )
  - [Homebrew](http://braumeister.org/formula/buku) for OS X, or its Linux fork, [Linuxbrew](https://github.com/Linuxbrew/linuxbrew/blob/master/Library/Formula/buku.rb)
 
+## Debian package
+
+If you are on a Debian (including Ubuntu) based system visit [the latest stable release](https://github.com/jarun/Buku/releases/latest) and download the`.deb`package. To install, run:
+
+    $ sudo dpkg -i buku-$version-all.deb
+
+Please substitute `$version` with the appropriate package version.
+
 # Usage
 
 ## Cmdline options
 
 **NOTE:** If you are using `buku` v1.9 or below please refer to the installed man page or program help.
 
-    usage: buku [-a URL [tags ...]] [-u [N]] [-d [N]] [-h]
+    usage: buku [OPTIONS] [KEYWORD [KEYWORD ...]]
-                [--url keyword] [--tag [...]] [-t [...]] [-c [...]]
-                [-s keyword [...]] [-S keyword [...]] [--st [...]]
-                [-k [N]] [-l [N]] [-p [N]] [-f N] [-r oldtag [newtag ...]]
-                [-j] [-e file] [-i file] [-m file] [--noprompt] [-o N]
 
     A private command-line bookmark manager. Your mini web!
 
@@ -131,8 +136,9 @@ Shell completion scripts for Bash, Fish and Zsh can be found in respective subdi
 
     edit options:
       --url keyword        specify url, works with -u only
-      --tag [...]          set comma-separated tags, works with -a, -u
+      --tag [+] [...]      set comma-separated tags, works with -a, -u
                            clears tags, if no arguments
+                           appends tags, if preceded by '+'
       -t, --title [...]    manually set title, works with -a, -u
                            if no arguments:
                            -a: do not set title, -u: clear title
@@ -147,7 +153,7 @@ Shell completion scripts for Bash, Fish and Zsh can be found in respective subdi
                            special keyword -
                            "blank": list entries with empty title/tag
       --st, --stag [...]   search bookmarks by tag
-                           list all tags alphabetically, if no arguments
+                           list tags alphabetically, if no arguments
 
     encryption options:
       -l, --lock [N]       encrypt DB file with N (> 0, default 8)
@@ -286,13 +292,16 @@ The same number of iterations must be used for one lock & unlock instance. Defau
 20. **Delete tag** 'old tag' from DB:
 
         $ buku -r 'old tag'
-21. **Open URL** at index 15012014 in browser:
+21. **Append tags** 'tag 1', 'tag 2' to existing tags of bookmark at index 15012014:
+
+        $ buku -u 15012014 --tag + tag 1, tag 2
+22. **Open URL** at index 15012014 in browser:
 
         $ buku -o 15012014
-22. To list bookmarks with no title or tags for **bookkeeping**:
+23. To list bookmarks with no title or tags for **bookkeeping**:
 
         $ buku -S blank
-23. More **help**:
+24. More **help**:
 
         $ buku
         $ man buku

auto-completion/fish/buku.fish

@@ -20,7 +20,7 @@ complete -c buku -s r -l replace -r --description 'replace a tag'
 complete -c buku -s s -l sany    -r --description 'search any keyword'
 complete -c buku -s S -l sall    -r --description 'search all keywords'
 complete -c buku -l st -l stag      --description 'search by tag or show tags'
-complete -c buku -l tag             --description 'set tags'
+complete -c buku -l tag             --description 'set tags, use + to append'
 complete -c buku -s t -l title      --description 'set custom title'
 complete -c buku -s u -l update     --description 'update bookmark'
 complete -c buku -l url             --description 'set url'

auto-completion/zsh/_buku

@@ -25,7 +25,7 @@ args=(
     '(-s --sany)'{-s,--sany}'[search any keyword]:keyword(s)'
     '(-s --sall)'{-s,--sall}'[search all keywords]:keyword(s)'
     '(-st --stag)'{--st,--stag}'[search by tag or show tags]'
-    '(--tag)'{--tag}'[set tags]'
+    '(--tag)'{--tag}'[set tags, use + to append]'
     '(-t --title)'{-t,--title}'[set custom title]'
     '(-u --update)'{-u,--update}'[update bookmark]'
     '(--url)'{--url}'[set url]'

buku

@@ -436,12 +436,37 @@ class BukuDb:
         except Exception as e:
             print('\x1b[1mEXCEPTION\x1b[21m [add_bookmark]: (%s) %s' % (type(e).__name__, e))
 
-    def update_bookmark(self, index, url='', title_manual=None, tag_manual=None, desc=None):
+    def append_tag_at_index(self, index, tag_manual):
+        """ Append tags for bookmark at index
+
+        :param index: int position of record, 0 for all
+        :tag_manual: string of comma-separated tags to add manually
+        """
+
+        if index == 0:
+            resp = input('Tags will be appended for ALL bookmarks. Enter \x1b[1my\x1b[21m to confirm: ')
+            if resp != 'y':
+                return
+
+            self.cur.execute('SELECT id, tags FROM bookmarks ORDER BY id ASC')
+        else:
+            self.cur.execute('SELECT id, tags FROM bookmarks WHERE id = ?', (index,))
+
+        resultset = self.cur.fetchall()
+        for row in resultset:
+            tags = '%s%s' % (row[1], tag_manual[1:])
+            tags = parse_tags([tags])
+            self.cur.execute('UPDATE bookmarks SET tags = ? WHERE id = ?', (tags, row[0],))
+
+        self.conn.commit()
+
+    def update_bookmark(self, index, url='', title_manual=None,
+                        tag_manual=None, desc=None, append_tag=False):
         """ Update an existing record at index
         Update all records if index is 0 and url is not specified.
         URL is an exception because URLs are unique in DB.
 
-        :param index: int position to update
+        :param index: int position to update, 0 for all
         :param url: address
         :param tag_manual: string of comma-separated tags to add manually
         :param title_manual: string title to add manually
@@ -461,6 +486,9 @@ class BukuDb:
 
         # Update tags if passed as argument
         if tag_manual is not None:
+            if append_tag:
+                self.append_tag_at_index(index, tag_manual)
+            else:
                 query = '%s tags = ?,' % query
                 arguments += (tag_manual,)
                 to_update = True
@@ -487,7 +515,7 @@ class BukuDb:
                 print('\x1B[91mTitle: []\x1B[0m')
             elif debug:
                 print('Title: [%s]' % meta)
-        elif not to_update:
+        elif not to_update and not append_tag:
             self.refreshdb(index)
             if index > 0:
                 self.print_bookmark(index)
@@ -668,7 +696,8 @@ class BukuDb:
         Print only bookmarks with blank title or tag if empty is True
         Note: URL is printed on top because title may be blank
 
-        Params: index to print, flag to show only bookmarks with no title or tags
+        Params: index to print (0 for all)
+	        empty flag to show only bookmarks with no title or tags
         """
 
         if index == 0:  # Show all entries
@@ -882,15 +911,14 @@ class BukuDb:
         Params: Path to file to merge
         """
 
-        if not os.path.exists(fp):
-            printmsg('%s not found' % fp, 'ERROR')
-            return
-
         try:
-            # Create a connection
+            # Connect to input DB
-            connfp = sqlite3.connect(fp)
+            if sys.version_info >= (3,4,4):
                 # Python 3.4.4 and above
-            # connfp = sqlite3.connect('file:%s?mode=ro' % fp, uri=True)
+                connfp = sqlite3.connect('file:%s?mode=ro' % fp, uri=True)
+            else:
+                connfp = sqlite3.connect(fp)
+
             curfp = connfp.cursor()
         except Exception as e:
             print('\x1b[1mEXCEPTION\x1b[21m [mergedb]: (%s) %s' % (type(e).__name__, e))
@@ -983,6 +1011,7 @@ def connect_server(url, fullurl=False, forced=False):
 
     urlconn.request('GET', url, None, {
         'Accept-encoding': 'gzip',
+        'DNT': '1',
     })
     return (urlconn, urlconn.getresponse())
 
@@ -1417,11 +1446,7 @@ if __name__ == '__main__':
     argparser = ExtendedArgumentParser(
         description='A private command-line bookmark manager. Your mini web!',
         formatter_class=argparse.RawTextHelpFormatter,
-        usage='''buku [-a URL [tags ...]] [-u [N]] [-d [N]] [-h]
+        usage='''buku [OPTIONS] [KEYWORD [KEYWORD ...]]''',
-            [--url keyword] [--tag [...]] [-t [...]] [-c [...]]
-            [-s keyword [...]] [-S keyword [...]] [--st [...]]
-            [-k [N]] [-l [N]] [-p [N]] [-f N] [-r oldtag [newtag ...]]
-            [-j] [-e file] [-i file] [-m file] [--noprompt] [-o N]''',
         add_help=False
     )
 
@@ -1446,8 +1471,9 @@ if __name__ == '__main__':
     edit_group = argparser.add_argument_group(
         title='edit options',
         description='''--url keyword        specify url, works with -u only
---tag [...]          set comma-separated tags, works with -a, -u
+--tag [+] [...]      set comma-separated tags, works with -a, -u
                      clears tags, if no arguments
+                     appends tags, if preceded by '+'
 -t, --title [...]    manually set title, works with -a, -u
                      if no arguments:
                      -a: do not set title, -u: clear title
@@ -1468,7 +1494,7 @@ if __name__ == '__main__':
                      special keyword -
                      "blank": list entries with empty title/tag
 --st, --stag [...]   search bookmarks by tag
-                     list all tags alphabetically, if no arguments''')
+                     list tags alphabetically, if no arguments''')
     search_group.add_argument('-s', '--sany', nargs='+', metavar='keyword', help=argparse.SUPPRESS)
     search_group.add_argument('-S', '--sall', nargs='+', metavar='keyword', help=argparse.SUPPRESS)
     search_group.add_argument('--st', '--stag', nargs='*', dest='stag', action=CustomTagSearchAction, metavar='keyword', help=argparse.SUPPRESS)
@@ -1563,8 +1589,15 @@ if __name__ == '__main__':
         tags = DELIMITER
         keywords = args.addurl
         if tagManual is not None:
+            if tagManual[0] == '+' and len(tagManual) == 1:
+                pass
+            elif tagManual[0] == '+':
+                tagManual = tagManual[1:]
+                # In case of add, args.addurl may have URL followed by tags
                 # Add DELIMITER as url+tags may not end with comma
                 keywords = args.addurl + [DELIMITER] + tagManual
+            else:
+                keywords = args.addurl + [DELIMITER] + tagManual
 
         if len(keywords) > 1:
             tags = parse_tags(keywords[1:])
@@ -1586,10 +1619,18 @@ if __name__ == '__main__':
         else:
             new_url = ''
 
+        append = False
+        if tagManual is not None:
+            if tagManual[0] == '+' and len(tagManual) == 1:
+                tagManual = None
+            elif tagManual[0] == '+':
+                tagManual = tagManual[1:]
+                append = True
+
         # Parse tags into a comma-separated string
         tags = parse_tags(tagManual)
 
-        bdb.update_bookmark(update_index, new_url, titleManual, tags, description)
+        bdb.update_bookmark(update_index, new_url, titleManual, tags, description, append)
 
     # Delete record(s)
     if args.delete is not None:

buku.1

@@ -2,16 +2,7 @@
 .SH NAME
 buku \- A private command-line bookmark manager. Your mini web!
 .SH SYNOPSIS
-.B buku
+.B buku [OPTIONS] [KEYWORD [KEYWORD ...]]
-[-a URL [tags ...]] [-u [N]] [-d [N]] [-h]
-.br
-     [--url keyword] [--tag [...]] [-t [...]] [-c [...]]
-.br
-     [-s keyword [...]] [-S keyword [...]] [--st [...]]
-.br
-     [-k [N]] [-l [N]] [-p [N]] [-f N] [-r oldtag [newtag ...]]
-.br
-     [-j] [-e file] [-i file] [-m file] [--noprompt] [-o N]
 .SH DESCRIPTION
 .B buku
 is a command-line tool to save, tag and search bookmarks.
@@ -78,8 +69,8 @@ Show program help and exit.
 .BI \--url " [...]"
 Specify the URL, works with -u only. Fetches and updates title if --title is not used.
 .TP
-.BI \--tag " [...]"
+.BI \--tag " [+] [...]"
-Specify comma separated tags, works with -a, -u. Clears the tags, if no arguments passed.
+Specify comma separated tags, works with -a, -u. Clears the tags, if no arguments passed. Appends tags, if list of tags is preceded by '+'.
 .TP
 .BI \-t " " \--title " [...]"
 Manually specify the title, works with -a, -u. Omits or clears the title, if no arguments passed.
@@ -355,13 +346,20 @@ The same number of iterations must be used for one lock & unlock instance. Defau
 .B buku -r 'old tag'
 .PP
 .IP 21. 4
+\fBAppend tags\fR 'tag 1', 'tag 2' to existing tags of bookmark at index 15012014:
+.PP
+.EX
+.IP
+.B buku -u 15012014 --tag + tag 1, tag 2
+.PP
+.IP 22. 4
 \fBOpen URL\fR at index 15012014 in browser:
 .PP
 .EX
 .IP
 .B buku -o 15012014
 .PP
-.IP 22. 4
+.IP 23. 4
 To list bookmarks with no title or tags for \fBbookkeeping\fR:
 .PP
 .EX