Merge remote-tracking branch 'upstream/master' into XMP_Metadata3

.github/ISSUE_TEMPLATE/bug_report.md

@@ -6,6 +6,7 @@ labels: ''
 assignees: ''
 
 ---
+<!-- Please have a look at our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md) -->
 
 **Describe the bug/problem**
 A clear and concise description of what the bug is. If you are asking for support, please check our [Wiki](https://github.com/janeczku/calibre-web/wiki) if your question is already answered there.
@@ -30,7 +31,7 @@ If applicable, add screenshots to help explain your problem.
  - OS: [e.g. Windows 10/Raspberry Pi OS]
  - Python version: [e.g. python2.7]
  - Calibre-Web version: [e.g. 0.6.8 or 087c4c59 (git rev-parse --short HEAD)]:
- - Docker container: [None/Technosoft2000/Linuxuser]:
+ - Docker container: [None/Technosoft2000/LinuxServer]:
  - Special Hardware: [e.g. Rasperry Pi Zero]
  - Browser: [e.g. Chrome 83.0.4103.97, Safari 13.3.7, Firefox 68.0.1 ESR]
 

.github/ISSUE_TEMPLATE/feature_request.md

@@ -7,6 +7,8 @@ assignees: ''
 
 ---
 
+<!-- Please have a look at our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md) -->
+
 **Is your feature request related to a problem? Please describe.**
 A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
 

.gitignore

@@ -10,6 +10,7 @@ env/
 venv/
 eggs/
 dist/
+executable/
 build/
 vendor/
 .eggs/

README.md

@@ -12,7 +12,7 @@ Calibre-Web is a web app providing a clean interface for browsing, reading and d
 - full graphical setup
 - User management with fine-grained per-user permissions
 - Admin interface
-- User Interface in czech, dutch, english, finnish, french, german, hungarian, italian, japanese, khmer, polish, russian, simplified chinese, spanish, swedish, turkish, ukrainian
+- User Interface in czech, dutch, english, finnish, french, german, greek, hungarian, italian, japanese, khmer, polish, russian, simplified chinese, spanish, swedish, turkish, ukrainian
 - OPDS feed for eBook reader apps 
 - Filter and search by titles, authors, tags, series and language
 - Create a custom book collection (shelves)
@@ -32,8 +32,8 @@ Calibre-Web is a web app providing a clean interface for browsing, reading and d
 
 ## Quick start
 
-1. Install dependencies by running `pip3 install --target vendor -r requirements.txt` (python3.x) or `pip install --target vendor -r requirements.txt` (python2.7).
-2. Execute the command: `python cps.py` (or `nohup python cps.py` - recommended if you want to exit the terminal window)
+1. Install dependencies by running `pip3 install --target vendor -r requirements.txt` (python3.x). Alternativly set up a python virtual environment.
+2. Execute the command: `python3 cps.py` (or `nohup python3 cps.py` - recommended if you want to exit the terminal window)
 3. Point your browser to `http://localhost:8083` or `http://localhost:8083/opds` for the OPDS catalog
 4. Set `Location of Calibre database` to the path of the folder where your Calibre library (metadata.db) lives, push "submit" button\
    Optionally a Google Drive can be used to host the calibre library [-> Using Google Drive integration](https://github.com/janeczku/calibre-web/wiki/Configuration#using-google-drive-integration)
@@ -48,27 +48,27 @@ Please note that running the above install command can fail on some versions of
 
 ## Requirements
 
-python 3.x+, (Python 2.7+)
+python 3.x+
 
 Optionally, to enable on-the-fly conversion from one ebook format to another when using the send-to-kindle feature, or during editing of ebooks metadata:
 
 [Download and install](https://calibre-ebook.com/download) the Calibre desktop program for your platform and enter the folder including program name (normally /opt/calibre/ebook-convert, or C:\Program Files\calibre\ebook-convert.exe) in the field "calibre's converter tool" on the setup page.
 
-[Download](https://github.com/geek1011/kepubify/releases/tag/v3.1.2) Kepubify tool for your platform and place the binary starting with `kepubify` in Linux: `\opt\kepubify` Windows: `C:\Program Files\kepubify`.
+[Download](https://github.com/pgaskin/kepubify/releases/latest) Kepubify tool for your platform and place the binary starting with `kepubify` in Linux: `\opt\kepubify` Windows: `C:\Program Files\kepubify`.
 
 ## Docker Images
 
 Pre-built Docker images are available in these Docker Hub repositories:
 
 #### **Technosoft2000 - x64**
-+ Docker Hub - [https://hub.docker.com/r/technosoft2000/calibre-web/](https://hub.docker.com/r/technosoft2000/calibre-web/)
++ Docker Hub - [https://hub.docker.com/r/technosoft2000/calibre-web](https://hub.docker.com/r/technosoft2000/calibre-web)
 + Github - [https://github.com/Technosoft2000/docker-calibre-web](https://github.com/Technosoft2000/docker-calibre-web) 
 
     Includes the Calibre `ebook-convert` binary.
     + The "path to convertertool" should be set to `/opt/calibre/ebook-convert`
 
 #### **LinuxServer - x64, armhf, aarch64**
-+ Docker Hub - [https://hub.docker.com/r/linuxserver/calibre-web/](https://hub.docker.com/r/linuxserver/calibre-web/)
++ Docker Hub - [https://hub.docker.com/r/linuxserver/calibre-web](https://hub.docker.com/r/linuxserver/calibre-web)
 + Github - [https://github.com/linuxserver/docker-calibre-web](https://github.com/linuxserver/docker-calibre-web)
 + Github - (Optional Calibre layer) - [https://github.com/linuxserver/docker-calibre-web/tree/calibre](https://github.com/linuxserver/docker-calibre-web/tree/calibre) 
 
@@ -83,3 +83,7 @@ Pre-built Docker images are available in these Docker Hub repositories:
 # Wiki
 
 For further information, How To's and FAQ please check the [Wiki](https://github.com/janeczku/calibre-web/wiki)
+
+# Contributing to Calibre-Web
+
+Please have a look at our [Contributing Guidelines](https://github.com/janeczku/calibre-web/blob/master/CONTRIBUTING.md) 

cps.py

@@ -41,6 +41,8 @@ from cps.shelf import shelf
 from cps.admin import admi
 from cps.gdrive import gdrive
 from cps.editbooks import editbook
+from cps.remotelogin import remotelogin
+from cps.error_handler import init_errorhandler
 
 try:
     from cps.kobo import kobo, get_kobo_activated
@@ -58,12 +60,17 @@ except ImportError:
 
 def main():
     app = create_app()
+
+    init_errorhandler()
+
     app.register_blueprint(web)
     app.register_blueprint(opds)
     app.register_blueprint(jinjia)
     app.register_blueprint(about)
     app.register_blueprint(shelf)
     app.register_blueprint(admi)
+    app.register_blueprint(remotelogin)
+    # if config.config_use_google_drive:
     app.register_blueprint(gdrive)
     app.register_blueprint(editbook)
     if kobo_available:

cps/__init__.py

@@ -45,6 +45,7 @@ mimetypes.add_type('application/fb2+zip', '.fb2')
 mimetypes.add_type('application/x-mobipocket-ebook', '.mobi')
 mimetypes.add_type('application/x-mobipocket-ebook', '.prc')
 mimetypes.add_type('application/vnd.amazon.ebook', '.azw')
+mimetypes.add_type('application/x-mobi8-ebook', '.azw3')
 mimetypes.add_type('application/x-cbr', '.cbr')
 mimetypes.add_type('application/x-cbz', '.cbz')
 mimetypes.add_type('application/x-cbt', '.cbt')
@@ -73,7 +74,6 @@ ub.init_db(cli.settingspath)
 # pylint: disable=no-member
 config = config_sql.load_configuration(ub.session)
 
-searched_ids = {}
 web_server = WebServer()
 
 babel = Babel()
@@ -83,6 +83,8 @@ log = logger.create()
 
 from . import services
 
+db.CalibreDB.setup_db(config, cli.settingspath)
+
 calibre_db = db.CalibreDB()
 
 def create_app():
@@ -91,18 +93,20 @@ def create_app():
     if sys.version_info < (3, 0):
         app.static_folder = app.static_folder.decode('utf-8')
         app.root_path = app.root_path.decode('utf-8')
-        app.instance_path = app.instance_path .decode('utf-8')
+        app.instance_path = app.instance_path.decode('utf-8')
 
-    cache_buster.init_cache_busting(app)
+    if os.environ.get('FLASK_DEBUG'):
+    	cache_buster.init_cache_busting(app)
 
     log.info('Starting Calibre Web...')
+    if sys.version_info < (3, 0):
+        log.info('Python2 is EOL since end of 2019, this version of Calibre-Web supporting Python2 please consider upgrading to Python3')
+        print('Python2 is EOL since end of 2019, this version of Calibre-Web supporting Python2 please consider upgrading to Python3')
     Principal(app)
     lm.init_app(app)
     app.secret_key = os.getenv('SECRET_KEY', config_sql.get_flask_session_key(ub.session))
 
     web_server.init_app(app, config)
-    calibre_db.setup_db(config, cli.settingspath)
-    calibre_db.start()
 
     babel.init_app(app)
     _BABEL_TRANSLATIONS.update(str(item) for item in babel.list_translations())

cps/about.py

@@ -31,7 +31,7 @@ import werkzeug, flask, flask_login, flask_principal, jinja2
 from flask_babel import gettext as _
 
 from . import db, calibre_db, converter, uploader, server, isoLanguages, constants
-from .web import render_title_template
+from .render_template import render_title_template
 try:
     from flask_login import __version__ as flask_loginVersion
 except ImportError:
@@ -48,6 +48,11 @@ try:
 except ImportError:
     flask_danceVersion = None
 
+try:
+    from greenlet import __version__ as greenlet_Version
+except ImportError:
+    greenlet_Version = None
+
 from . import services
 
 about = flask.Blueprint('about', __name__)
@@ -77,11 +82,18 @@ _VERSIONS = OrderedDict(
     python_LDAP = services.ldapVersion if bool(services.ldapVersion) else None,
     Goodreads = u'installed' if bool(services.goodreads_support) else None,
     jsonschema = services.SyncToken.__version__  if bool(services.SyncToken) else None,
-    flask_dance = flask_danceVersion
+    flask_dance = flask_danceVersion,
+    greenlet = greenlet_Version
 )
 _VERSIONS.update(uploader.get_versions())
 
 
+def collect_stats():
+    _VERSIONS['ebook converter'] = _(converter.get_calibre_version())
+    _VERSIONS['unrar'] = _(converter.get_unrar_version())
+    _VERSIONS['kepubify'] = _(converter.get_kepubify_version())
+    return _VERSIONS
+
 @about.route("/stats")
 @flask_login.login_required
 def stats():
@@ -89,8 +101,7 @@ def stats():
     authors = calibre_db.session.query(db.Authors).count()
     categorys = calibre_db.session.query(db.Tags).count()
     series = calibre_db.session.query(db.Series).count()
-    _VERSIONS['ebook converter'] = _(converter.get_calibre_version())
-    _VERSIONS['unrar'] = _(converter.get_unrar_version())
-    _VERSIONS['kepubify'] = _(converter.get_kepubify_version())
-    return render_title_template('stats.html', bookcounter=counter, authorcounter=authors, versions=_VERSIONS,
+    return render_title_template('stats.html', bookcounter=counter, authorcounter=authors, versions=collect_stats(),
                                  categorycounter=categorys, seriecounter=series, title=_(u"Statistics"), page="stat")
+
+

cps/cli.py

@@ -45,6 +45,7 @@ parser.add_argument('-v', '--version', action='version', help='Shows version num
                     version=version_info())
 parser.add_argument('-i', metavar='ip-address', help='Server IP-Address to listen')
 parser.add_argument('-s', metavar='user:pass', help='Sets specific username to new password')
+parser.add_argument('-f', action='store_true', help='Enables filepicker in unconfigured mode')
 args = parser.parse_args()
 
 if sys.version_info < (3, 0):
@@ -109,4 +110,10 @@ if ipadress:
         sys.exit(1)
 
 # handle and check user password argument
-user_password = args.s or None
+user_credentials = args.s or None
+if user_credentials and ":" not in user_credentials:
+    print("No valid username:password format")
+    sys.exit(3)
+
+# Handles enableing of filepicker
+filepicker = args.f or None

cps/comic.py

@@ -18,27 +18,27 @@
 
 from __future__ import division, print_function, unicode_literals
 import os
-import io
 
 from . import logger, isoLanguages
 from .constants import BookMeta
 
-try:
-    from PIL import Image as PILImage
-    use_PIL = True
-except ImportError as e:
-    use_PIL = False
-
 
 log = logger.create()
 
 
+try:
+    from wand.image import Image
+    use_IM = True
+except (ImportError, RuntimeError) as e:
+    use_IM = False
+
+
 try:
     from comicapi.comicarchive import ComicArchive, MetaDataStyle
     use_comic_meta = True
     try:
         from comicapi import __version__ as comic_version
-    except (ImportError):
+    except ImportError:
         comic_version = ''
 except (ImportError, LookupError) as e:
     log.debug('Cannot import comicapi, extracting comic metadata will not work: %s', e)
@@ -47,82 +47,88 @@ except (ImportError, LookupError) as e:
     try:
         import rarfile
         use_rarfile = True
-    except ImportError as e:
+    except (ImportError, SyntaxError) as e:
         log.debug('Cannot import rarfile, extracting cover files from rar files will not work: %s', e)
         use_rarfile = False
     use_comic_meta = False
 
-def _cover_processing(tmp_file_name, img, extension):
-    if use_PIL:
-        # convert to jpg because calibre only supports jpg
-        if extension in ('.png',  '.webp'):
-            imgc = PILImage.open(io.BytesIO(img))
-            im = imgc.convert('RGB')
-            tmp_bytesio = io.BytesIO()
-            im.save(tmp_bytesio, format='JPEG')
-            img = tmp_bytesio.getvalue()
+NO_JPEG_EXTENSIONS = ['.png', '.webp', '.bmp']
+COVER_EXTENSIONS = ['.png', '.webp', '.bmp', '.jpg', '.jpeg']
 
-    prefix = os.path.dirname(tmp_file_name)
-    if img:
-        tmp_cover_name = prefix + '/cover.jpg'
-        image = open(tmp_cover_name, 'wb')
-        image.write(img)
-        image.close()
-    else:
-        tmp_cover_name = None
+def _cover_processing(tmp_file_name, img, extension):
+    tmp_cover_name = os.path.join(os.path.dirname(tmp_file_name), 'cover.jpg')
+    if use_IM:
+        # convert to jpg because calibre only supports jpg
+        if extension in NO_JPEG_EXTENSIONS:
+            with Image(filename=tmp_file_name) as imgc:
+                imgc.format = 'jpeg'
+                imgc.transform_colorspace('rgb')
+                imgc.save(tmp_cover_name)
+                return tmp_cover_name
+
+    if not img:
+        return None
+
+    with open(tmp_cover_name, 'wb') as f:
+        f.write(img)
     return tmp_cover_name
 
 
-
-def _extractCover(tmp_file_name, original_file_extension, rarExceutable):
-    cover_data = extension = None
-    if use_comic_meta:
-        archive = ComicArchive(tmp_file_name)
-        for index, name in enumerate(archive.getPageNameList()):
+def _extract_Cover_from_archive(original_file_extension, tmp_file_name, rarExecutable):
+    cover_data = None
+    if original_file_extension.upper() == '.CBZ':
+        cf = zipfile.ZipFile(tmp_file_name)
+        for name in cf.namelist():
             ext = os.path.splitext(name)
             if len(ext) > 1:
                 extension = ext[1].lower()
-                if extension in ('.jpg', '.jpeg', '.png', '.webp'):
-                    cover_data = archive.getPage(index)
+                if extension in COVER_EXTENSIONS:
+                    cover_data = cf.read(name)
                     break
-    else:
-        if original_file_extension.upper() == '.CBZ':
-            cf = zipfile.ZipFile(tmp_file_name)
-            for name in cf.namelist():
-                ext = os.path.splitext(name)
-                if len(ext) > 1:
-                    extension = ext[1].lower()
-                    if extension in ('.jpg', '.jpeg', '.png', '.webp'):
-                        cover_data = cf.read(name)
-                        break
-        elif original_file_extension.upper() == '.CBT':
-            cf = tarfile.TarFile(tmp_file_name)
+    elif original_file_extension.upper() == '.CBT':
+        cf = tarfile.TarFile(tmp_file_name)
+        for name in cf.getnames():
+            ext = os.path.splitext(name)
+            if len(ext) > 1:
+                extension = ext[1].lower()
+                if extension in COVER_EXTENSIONS:
+                    cover_data = cf.extractfile(name).read()
+                    break
+    elif original_file_extension.upper() == '.CBR' and use_rarfile:
+        try:
+            rarfile.UNRAR_TOOL = rarExecutable
+            cf = rarfile.RarFile(tmp_file_name)
             for name in cf.getnames():
                 ext = os.path.splitext(name)
                 if len(ext) > 1:
                     extension = ext[1].lower()
-                    if extension in ('.jpg', '.jpeg', '.png', '.webp'):
-                        cover_data = cf.extractfile(name).read()
+                    if extension in COVER_EXTENSIONS:
+                        cover_data = cf.read(name)
                         break
-        elif original_file_extension.upper() == '.CBR' and use_rarfile:
-            try:
-                rarfile.UNRAR_TOOL = rarExceutable
-                cf = rarfile.RarFile(tmp_file_name)
-                for name in cf.getnames():
-                    ext = os.path.splitext(name)
-                    if len(ext) > 1:
-                        extension = ext[1].lower()
-                        if extension in ('.jpg', '.jpeg', '.png', '.webp'):
-                            cover_data = cf.read(name)
-                            break
-            except Exception as e:
-                log.debug('Rarfile failed with error: %s', e)
+        except Exception as e:
+            log.debug('Rarfile failed with error: %s', e)
+    return cover_data
+
+
+def _extractCover(tmp_file_name, original_file_extension, rarExecutable):
+    cover_data = extension = None
+    if use_comic_meta:
+        archive = ComicArchive(tmp_file_name, rar_exe_path=rarExecutable)
+        for index, name in enumerate(archive.getPageNameList()):
+            ext = os.path.splitext(name)
+            if len(ext) > 1:
+                extension = ext[1].lower()
+                if extension in COVER_EXTENSIONS:
+                    cover_data = archive.getPage(index)
+                    break
+    else:
+        cover_data = _extract_Cover_from_archive(original_file_extension, tmp_file_name, rarExecutable)
     return _cover_processing(tmp_file_name, cover_data, extension)
 
 
-def get_comic_info(tmp_file_path, original_file_name, original_file_extension, rarExceutable):
+def get_comic_info(tmp_file_path, original_file_name, original_file_extension, rarExecutable):
     if use_comic_meta:
-        archive = ComicArchive(tmp_file_path, rar_exe_path=rarExceutable)
+        archive = ComicArchive(tmp_file_path, rar_exe_path=rarExecutable)
         if archive.seemsToBeAComicArchive():
             if archive.hasMetadata(MetaDataStyle.CIX):
                 style = MetaDataStyle.CIX
@@ -134,21 +140,16 @@ def get_comic_info(tmp_file_path, original_file_name, original_file_extension, r
             # if style is not None:
             loadedMetadata = archive.readMetadata(style)
 
-            lang = loadedMetadata.language
-            if lang:
-                if len(lang) == 2:
-                     loadedMetadata.language = isoLanguages.get(part1=lang).name
-                elif len(lang) == 3:
-                     loadedMetadata.language = isoLanguages.get(part3=lang).name
-            else:
-                 loadedMetadata.language = ""
+            lang = loadedMetadata.language or ""
+            loadedMetadata.language = isoLanguages.get_lang3(lang)
 
             return BookMeta(
                 file_path=tmp_file_path,
                 extension=original_file_extension,
                 title=loadedMetadata.title or original_file_name,
-                author=" & ".join([credit["person"] for credit in loadedMetadata.credits if credit["role"] == "Writer"]) or u'Unknown',
-                cover=_extractCover(tmp_file_path, original_file_extension, rarExceutable),
+                author=" & ".join([credit["person"]
+                                   for credit in loadedMetadata.credits if credit["role"] == "Writer"]) or u'Unknown',
+                cover=_extractCover(tmp_file_path, original_file_extension, rarExecutable),
                 description=loadedMetadata.comments or "",
                 tags="",
                 series=loadedMetadata.series or "",
@@ -160,7 +161,7 @@ def get_comic_info(tmp_file_path, original_file_name, original_file_extension, r
         extension=original_file_extension,
         title=original_file_name,
         author=u'Unknown',
-        cover=_extractCover(tmp_file_path, original_file_extension, rarExceutable),
+        cover=_extractCover(tmp_file_path, original_file_extension, rarExecutable),
         description="",
         tags="",
         series="",

cps/config_sql.py

@@ -19,10 +19,10 @@
 
 from __future__ import division, print_function, unicode_literals
 import os
-import json
 import sys
 
 from sqlalchemy import exc, Column, String, Integer, SmallInteger, Boolean, BLOB, JSON
+from sqlalchemy.exc import OperationalError
 from sqlalchemy.ext.declarative import declarative_base
 
 from . import constants, cli, logger, ub
@@ -109,9 +109,12 @@ class _Settings(_Base):
     config_ldap_serv_username = Column(String, default='cn=admin,dc=example,dc=org')
     config_ldap_serv_password = Column(String, default="")
     config_ldap_encryption = Column(SmallInteger, default=0)
+    config_ldap_cacert_path = Column(String, default="")
     config_ldap_cert_path = Column(String, default="")
+    config_ldap_key_path = Column(String, default="")
     config_ldap_dn = Column(String, default='dc=example,dc=org')
     config_ldap_user_object = Column(String, default='uid=%s')
+    config_ldap_member_user_object = Column(String, default='') #
     config_ldap_openldap = Column(Boolean, default=True)
     config_ldap_group_object_filter = Column(String, default='(&(objectclass=posixGroup)(cn=%s))')
     config_ldap_group_members_field = Column(String, default='memberUid')
@@ -143,15 +146,16 @@ class _ConfigSQL(object):
         self.load()
 
         change = False
-        if self.config_converterpath == None:
+        if self.config_converterpath == None:  # pylint: disable=access-member-before-definition
             change = True
             self.config_converterpath = autodetect_calibre_binary()
 
-        if self.config_kepubifypath == None:
+        if self.config_kepubifypath == None:  # pylint: disable=access-member-before-definition
+
             change = True
             self.config_kepubifypath = autodetect_kepubify_binary()
 
-        if self.config_rarfile_location == None:
+        if self.config_rarfile_location == None:  # pylint: disable=access-member-before-definition
             change = True
             self.config_rarfile_location = autodetect_unrar_binary()
         if change:
@@ -178,7 +182,8 @@ class _ConfigSQL(object):
             return None
         return self.config_keyfile
 
-    def get_config_ipaddress(self):
+    @staticmethod
+    def get_config_ipaddress():
         return cli.ipadress or ""
 
     def _has_role(self, role_flag):
@@ -269,6 +274,14 @@ class _ConfigSQL(object):
         setattr(self, field, new_value)
         return True
 
+    def toDict(self):
+        storage = {}
+        for k, v in self.__dict__.items():
+            if k[0] != '_' and not k.endswith("password") and not k.endswith("secret"):
+                storage[k] = v
+        return storage
+
+
     def load(self):
         '''Load all configuration values from the underlying storage.'''
         s = self._read_from_storage()  # type: _Settings
@@ -287,13 +300,18 @@ class _ConfigSQL(object):
                 db_file = os.path.join(self.config_calibre_dir, 'metadata.db')
                 have_metadata_db = os.path.isfile(db_file)
         self.db_configured = have_metadata_db
-        constants.EXTENSIONS_UPLOAD = [x.lstrip().rstrip() for x in self.config_upload_formats.split(',')]
+        constants.EXTENSIONS_UPLOAD = [x.lstrip().rstrip().lower() for x in self.config_upload_formats.split(',')]
+        # pylint: disable=access-member-before-definition
         logfile = logger.setup(self.config_logfile, self.config_log_level)
         if logfile != self.config_logfile:
             log.warning("Log path %s not valid, falling back to default", self.config_logfile)
             self.config_logfile = logfile
             self._session.merge(s)
-            self._session.commit()
+            try:
+                self._session.commit()
+            except OperationalError as e:
+                log.error('Database error: %s', e)
+                self._session.rollback()
 
     def save(self):
         '''Apply all configuration values to the underlying storage.'''
@@ -307,7 +325,11 @@ class _ConfigSQL(object):
 
         log.debug("_ConfigSQL updating storage")
         self._session.merge(s)
-        self._session.commit()
+        try:
+            self._session.commit()
+        except OperationalError as e:
+            log.error('Database error: %s', e)
+            self._session.rollback()
         self.load()
 
     def invalidate(self, error=None):
@@ -348,7 +370,10 @@ def _migrate_table(session, orm_class):
                 changed = True
 
     if changed:
-        session.commit()
+        try:
+            session.commit()
+        except OperationalError:
+            session.rollback()
 
 
 def autodetect_calibre_binary():

cps/constants.py

@@ -21,7 +21,9 @@ import sys
 import os
 from collections import namedtuple
 
+# if installed via pip this variable is set to true
 HOME_CONFIG = False
+UPDATER_AVAILABLE = True
 
 # Base dir is parent of current file, necessary if called from different folder
 if sys.version_info < (3, 0):
@@ -40,7 +42,7 @@ if HOME_CONFIG:
         os.makedirs(home_dir)
     CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', home_dir)
 else:
-    CONFIG_DIR      = os.environ.get('CALIBRE_DBPATH', BASE_DIR)
+    CONFIG_DIR = os.environ.get('CALIBRE_DBPATH', BASE_DIR)
 
 
 ROLE_USER               = 0 << 0
@@ -81,10 +83,11 @@ SIDEBAR_PUBLISHER       = 1 << 12
 SIDEBAR_RATING          = 1 << 13
 SIDEBAR_FORMAT          = 1 << 14
 SIDEBAR_ARCHIVED        = 1 << 15
-# SIDEBAR_LIST            = 1 << 16
+SIDEBAR_DOWNLOAD        = 1 << 16
+SIDEBAR_LIST            = 1 << 17
 
 ADMIN_USER_ROLES        = sum(r for r in ALL_ROLES.values()) & ~ROLE_ANONYMOUS
-ADMIN_USER_SIDEBAR      = (SIDEBAR_ARCHIVED << 1) - 1
+ADMIN_USER_SIDEBAR      = (SIDEBAR_LIST << 1) - 1
 
 UPDATE_STABLE       = 0 << 0
 AUTO_UPDATE_STABLE  = 1 << 0
@@ -101,7 +104,7 @@ LDAP_AUTH_SIMPLE         = 0
 
 DEFAULT_MAIL_SERVER = "mail.example.org"
 
-DEFAULT_PASSWORD    = "admin123"
+DEFAULT_PASSWORD    = "admin123"  # nosec  # noqa
 DEFAULT_PORT        = 8083
 env_CALIBRE_PORT = os.environ.get("CALIBRE_PORT", DEFAULT_PORT)
 try:
@@ -112,7 +115,8 @@ del env_CALIBRE_PORT
 
 
 EXTENSIONS_AUDIO    = {'mp3', 'mp4', 'ogg', 'opus', 'wav', 'flac', 'm4a', 'm4b'}
-EXTENSIONS_CONVERT  = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt']
+EXTENSIONS_CONVERT_FROM  = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt','cbz','cbr']
+EXTENSIONS_CONVERT_TO  = ['pdf', 'epub', 'mobi', 'azw3', 'docx', 'rtf', 'fb2', 'lit', 'lrf', 'txt', 'htmlz', 'rtf', 'odt']
 EXTENSIONS_UPLOAD   = {'txt', 'pdf', 'epub', 'kepub', 'mobi', 'azw', 'azw3', 'cbr', 'cbz', 'cbt', 'djvu', 'prc', 'doc', 'docx',
                        'fb2', 'html', 'rtf', 'lit', 'odt', 'mp3', 'mp4', 'ogg', 'opus', 'wav', 'flac', 'm4a', 'm4b'}
 
@@ -128,7 +132,7 @@ def selected_roles(dictionary):
 BookMeta = namedtuple('BookMeta', 'file_path, extension, title, author, cover, description, tags, series, '
                                   'series_id, languages')
 
-STABLE_VERSION = {'version': '0.6.9 Beta'}
+STABLE_VERSION = {'version': '0.6.12 Beta'}
 
 NIGHTLY_VERSION = {}
 NIGHTLY_VERSION[0] = '$Format:%H$'

cps/converter.py

@@ -19,7 +19,6 @@
 from __future__ import division, print_function, unicode_literals
 import os
 import re
-import sys
 from flask_babel import gettext as _
 
 from . import config, logger

cps/db.py

@@ -24,18 +24,17 @@ import re
 import ast
 import json
 from datetime import datetime
-import threading
 
 from sqlalchemy import create_engine
 from sqlalchemy import Table, Column, ForeignKey, CheckConstraint
 from sqlalchemy import String, Integer, Boolean, TIMESTAMP, Float
 from sqlalchemy.orm import relationship, sessionmaker, scoped_session
-from sqlalchemy.ext.declarative import declarative_base
-from sqlalchemy.exc import OperationalError
+from sqlalchemy.orm.collections import InstrumentedList
+from sqlalchemy.ext.declarative import declarative_base, DeclarativeMeta
 from sqlalchemy.pool import StaticPool
-from flask_login import current_user
 from sqlalchemy.sql.expression import and_, true, false, text, func, or_
 from sqlalchemy.ext.associationproxy import association_proxy
+from flask_login import current_user
 from babel import Locale as LC
 from babel.core import UnknownLocaleError
 from flask_babel import gettext as _
@@ -43,12 +42,15 @@ from flask_babel import gettext as _
 from . import logger, ub, isoLanguages
 from .pagination import Pagination
 
+from weakref import WeakSet
+
 try:
     import unidecode
     use_unidecode = True
 except ImportError:
     use_unidecode = False
 
+log = logger.create()
 
 cc_exceptions = ['datetime', 'comments', 'composite', 'series']
 cc_classes = {}
@@ -56,34 +58,34 @@ cc_classes = {}
 Base = declarative_base()
 
 books_authors_link = Table('books_authors_link', Base.metadata,
-    Column('book', Integer, ForeignKey('books.id'), primary_key=True),
-    Column('author', Integer, ForeignKey('authors.id'), primary_key=True)
-    )
+                           Column('book', Integer, ForeignKey('books.id'), primary_key=True),
+                           Column('author', Integer, ForeignKey('authors.id'), primary_key=True)
+                           )
 
 books_tags_link = Table('books_tags_link', Base.metadata,
-    Column('book', Integer, ForeignKey('books.id'), primary_key=True),
-    Column('tag', Integer, ForeignKey('tags.id'), primary_key=True)
-    )
+                        Column('book', Integer, ForeignKey('books.id'), primary_key=True),
+                        Column('tag', Integer, ForeignKey('tags.id'), primary_key=True)
+                        )
 
 books_series_link = Table('books_series_link', Base.metadata,
-    Column('book', Integer, ForeignKey('books.id'), primary_key=True),
-    Column('series', Integer, ForeignKey('series.id'), primary_key=True)
-    )
+                          Column('book', Integer, ForeignKey('books.id'), primary_key=True),
+                          Column('series', Integer, ForeignKey('series.id'), primary_key=True)
+                          )
 
 books_ratings_link = Table('books_ratings_link', Base.metadata,
-    Column('book', Integer, ForeignKey('books.id'), primary_key=True),
-    Column('rating', Integer, ForeignKey('ratings.id'), primary_key=True)
-    )
+                           Column('book', Integer, ForeignKey('books.id'), primary_key=True),
+                           Column('rating', Integer, ForeignKey('ratings.id'), primary_key=True)
+                           )
 
 books_languages_link = Table('books_languages_link', Base.metadata,
-    Column('book', Integer, ForeignKey('books.id'), primary_key=True),
-    Column('lang_code', Integer, ForeignKey('languages.id'), primary_key=True)
-    )
+                             Column('book', Integer, ForeignKey('books.id'), primary_key=True),
+                             Column('lang_code', Integer, ForeignKey('languages.id'), primary_key=True)
+                             )
 
 books_publishers_link = Table('books_publishers_link', Base.metadata,
-    Column('book', Integer, ForeignKey('books.id'), primary_key=True),
-    Column('publisher', Integer, ForeignKey('publishers.id'), primary_key=True)
-    )
+                              Column('book', Integer, ForeignKey('books.id'), primary_key=True),
+                              Column('publisher', Integer, ForeignKey('publishers.id'), primary_key=True)
+                              )
 
 
 class Identifiers(Base):
@@ -117,6 +119,12 @@ class Identifiers(Base):
             return u"Google Books"
         elif format_type == "kobo":
             return u"Kobo"
+        elif format_type == "litres":
+            return u"ЛитРес"
+        elif format_type == "issn":
+            return u"ISSN"
+        elif format_type == "isfdb":
+            return u"ISFDB"
         if format_type == "lubimyczytac":
             return u"Lubimyczytac"
         else:
@@ -125,9 +133,9 @@ class Identifiers(Base):
     def __repr__(self):
         format_type = self.type.lower()
         if format_type == "amazon" or format_type == "asin":
-            return u"https://amzn.com/{0}".format(self.val)
+            return u"https://amazon.com/dp/{0}".format(self.val)
         elif format_type.startswith('amazon_'):
-            return u"https://amazon.{0}/{1}".format(format_type[7:], self.val)
+            return u"https://amazon.{0}/dp/{1}".format(format_type[7:], self.val)
         elif format_type == "isbn":
             return u"https://www.worldcat.org/isbn/{0}".format(self.val)
         elif format_type == "doi":
@@ -141,11 +149,15 @@ class Identifiers(Base):
         elif format_type == "kobo":
             return u"https://www.kobo.com/ebook/{0}".format(self.val)
         elif format_type == "lubimyczytac":
-            return u" https://lubimyczytac.pl/ksiazka/{0}".format(self.val)
-        elif format_type == "url":
-            return u"{0}".format(self.val)
+            return u"https://lubimyczytac.pl/ksiazka/{0}/ksiazka".format(self.val)
+        elif format_type == "litres":
+            return u"https://www.litres.ru/{0}".format(self.val)
+        elif format_type == "issn":
+            return u"https://portal.issn.org/resource/ISSN/{0}".format(self.val)
+        elif format_type == "isfdb":
+            return u"http://www.isfdb.org/cgi-bin/pl.cgi?{0}".format(self.val)
         else:
-            return u""
+            return u"{0}".format(self.val)
 
 
 class Comments(Base):
@@ -159,6 +171,9 @@ class Comments(Base):
         self.text = text
         self.book = book
 
+    def get(self):
+        return self.text
+
     def __repr__(self):
         return u"<Comments({0})>".format(self.text)
 
@@ -172,6 +187,9 @@ class Tags(Base):
     def __init__(self, name):
         self.name = name
 
+    def get(self):
+        return self.name
+
     def __repr__(self):
         return u"<Tags('{0})>".format(self.name)
 
@@ -189,6 +207,9 @@ class Authors(Base):
         self.sort = sort
         self.link = link
 
+    def get(self):
+        return self.name
+
     def __repr__(self):
         return u"<Authors('{0},{1}{2}')>".format(self.name, self.sort, self.link)
 
@@ -204,6 +225,9 @@ class Series(Base):
         self.name = name
         self.sort = sort
 
+    def get(self):
+        return self.name
+
     def __repr__(self):
         return u"<Series('{0},{1}')>".format(self.name, self.sort)
 
@@ -217,6 +241,9 @@ class Ratings(Base):
     def __init__(self, rating):
         self.rating = rating
 
+    def get(self):
+        return self.rating
+
     def __repr__(self):
         return u"<Ratings('{0}')>".format(self.rating)
 
@@ -230,6 +257,12 @@ class Languages(Base):
     def __init__(self, lang_code):
         self.lang_code = lang_code
 
+    def get(self):
+        if self.language_name:
+            return self.language_name
+        else:
+            return self.lang_code
+
     def __repr__(self):
         return u"<Languages('{0}')>".format(self.lang_code)
 
@@ -245,13 +278,16 @@ class Publishers(Base):
         self.name = name
         self.sort = sort
 
+    def get(self):
+        return self.name
+
     def __repr__(self):
         return u"<Publishers('{0},{1}')>".format(self.name, self.sort)
 
 
 class Data(Base):
     __tablename__ = 'data'
-    __table_args__ = {'schema':'calibre'}
+    __table_args__ = {'schema': 'calibre'}
 
     id = Column(Integer, primary_key=True)
     book = Column(Integer, ForeignKey('books.id'), nullable=False)
@@ -265,6 +301,10 @@ class Data(Base):
         self.uncompressed_size = uncompressed_size
         self.name = name
 
+    # ToDo: Check
+    def get(self):
+        return self.name
+
     def __repr__(self):
         return u"<Data('{0},{1}{2}{3}')>".format(self.book, self.format, self.uncompressed_size, self.name)
 
@@ -272,14 +312,14 @@ class Data(Base):
 class Books(Base):
     __tablename__ = 'books'
 
-    DEFAULT_PUBDATE = "0101-01-01 00:00:00+00:00"
+    DEFAULT_PUBDATE = datetime(101, 1, 1, 0, 0, 0, 0)  # ("0101-01-01 00:00:00+00:00")
 
     id = Column(Integer, primary_key=True, autoincrement=True)
     title = Column(String(collation='NOCASE'), nullable=False, default='Unknown')
     sort = Column(String(collation='NOCASE'))
     author_sort = Column(String(collation='NOCASE'))
     timestamp = Column(TIMESTAMP, default=datetime.utcnow)
-    pubdate = Column(String) # , default=datetime.utcnow)
+    pubdate = Column(TIMESTAMP, default=DEFAULT_PUBDATE)
     series_index = Column(String, nullable=False, default="1.0")
     last_modified = Column(TIMESTAMP, default=datetime.utcnow)
     path = Column(String, default="", nullable=False)
@@ -309,7 +349,8 @@ class Books(Base):
         self.series_index = series_index
         self.last_modified = last_modified
         self.path = path
-        self.has_cover = has_cover
+        self.has_cover = (has_cover != None)
+
 
     def __repr__(self):
         return u"<Books('{0},{1}{2}{3}{4}{5}{6}{7}{8}')>".format(self.title, self.sort, self.author_sort,
@@ -320,6 +361,7 @@ class Books(Base):
     def atom_timestamp(self):
         return (self.timestamp.strftime('%Y-%m-%dT%H:%M:%S+00:00') or '')
 
+
 class Custom_Columns(Base):
     __tablename__ = 'custom_columns'
 
@@ -340,46 +382,73 @@ class Custom_Columns(Base):
         return display_dict
 
 
-class CalibreDB(threading.Thread):
+class AlchemyEncoder(json.JSONEncoder):
 
-    def __init__(self):
-        threading.Thread.__init__(self)
-        self.engine = None
-        self.session = None
-        self.queue = None
-        self.log = None
-        self.config = None
-
-    def add_queue(self,queue):
-        self.queue = queue
-        self.log = logger.create()
-
-    def run(self):
-        while True:
-            i = self.queue.get()
-            if i == 'dummy':
-                self.queue.task_done()
-                break
-            if i['task'] == 'add_format':
-                cur_book = self.session.query(Books).filter(Books.id == i['id']).first()
-                cur_book.data.append(i['format'])
+    def default(self, o):
+        if isinstance(o.__class__, DeclarativeMeta):
+            # an SQLAlchemy class
+            fields = {}
+            for field in [x for x in dir(o) if not x.startswith('_') and x != 'metadata']:
+                if field == 'books':
+                    continue
+                data = o.__getattribute__(field)
                 try:
-                    # db.session.merge(cur_book)
-                    self.session.commit()
-                except OperationalError as e:
-                    self.session.rollback()
-                    self.log.error("Database error: %s", e)
-                    # self._handleError(_(u"Database error: %(error)s.", error=e))
-                    # return
-            self.queue.task_done()
+                    if isinstance(data, str):
+                        data = data.replace("'", "\'")
+                    elif isinstance(data, InstrumentedList):
+                        el = list()
+                        for ele in data:
+                            if ele.get:
+                                el.append(ele.get())
+                            else:
+                                el.append(json.dumps(ele, cls=AlchemyEncoder))
+                        if field == 'authors':
+                            data = " & ".join(el)
+                        else:
+                            data = ",".join(el)
+                        if data == '[]':
+                            data = ""
+                    else:
+                        json.dumps(data)
+                    fields[field] = data
+                except Exception:
+                    fields[field] = ""
+            # a json-encodable dict
+            return fields
+
+        return json.JSONEncoder.default(self, o)
 
 
-    def stop(self):
-        self.queue.put('dummy')
+class CalibreDB():
+    _init = False
+    engine = None
+    config = None
+    session_factory = None
+    # This is a WeakSet so that references here don't keep other CalibreDB
+    # instances alive once they reach the end of their respective scopes
+    instances = WeakSet()
 
-    def setup_db(self, config, app_db_path):
-        self.config = config
-        self.dispose()
+    def __init__(self, expire_on_commit=True):
+        """ Initialize a new CalibreDB session
+        """
+        self.session = None
+        if self._init:
+            self.initSession(expire_on_commit)
+
+        self.instances.add(self)
+
+
+    def initSession(self, expire_on_commit=True):
+        self.session = self.session_factory()
+        self.session.expire_on_commit = expire_on_commit
+        self.update_title_sort(self.config)
+
+    @classmethod
+    def setup_db(cls, config, app_db_path):
+        cls.config = config
+        cls.dispose()
+
+        # toDo: if db changed -> delete shelfs, delete download books, delete read boks, kobo sync??
 
         if not config.config_calibre_dir:
             config.invalidate()
@@ -391,22 +460,21 @@ class CalibreDB(threading.Thread):
             return False
 
         try:
-            self.engine = create_engine('sqlite://',
-                                   echo=False,
-                                   isolation_level="SERIALIZABLE",
-                                   connect_args={'check_same_thread': False},
-                                   poolclass=StaticPool)
-            self.engine.execute("attach database '{}' as calibre;".format(dbpath))
-            self.engine.execute("attach database '{}' as app_settings;".format(app_db_path))
+            cls.engine = create_engine('sqlite://',
+                                       echo=False,
+                                       isolation_level="SERIALIZABLE",
+                                       connect_args={'check_same_thread': False},
+                                       poolclass=StaticPool)
+            cls.engine.execute("attach database '{}' as calibre;".format(dbpath))
+            cls.engine.execute("attach database '{}' as app_settings;".format(app_db_path))
 
-            conn = self.engine.connect()
+            conn = cls.engine.connect()
             # conn.text_factory = lambda b: b.decode(errors = 'ignore') possible fix for #1302
         except Exception as e:
             config.invalidate(e)
             return False
 
         config.db_configured = True
-        self.update_title_sort(config, conn.connection)
 
         if not cc_classes:
             cc = conn.execute("SELECT id, datatype FROM custom_columns")
@@ -421,12 +489,12 @@ class CalibreDB(threading.Thread):
                                      'book': Column(Integer, ForeignKey('books.id'),
                                                     primary_key=True),
                                      'map_value': Column('value', Integer,
-                                                     ForeignKey('custom_column_' +
-                                                                str(row.id) + '.id'),
-                                                     primary_key=True),
+                                                         ForeignKey('custom_column_' +
+                                                                    str(row.id) + '.id'),
+                                                         primary_key=True),
                                      'extra': Column(Float),
-                                     'asoc' : relationship('custom_column_' + str(row.id), uselist=False),
-                                     'value' : association_proxy('asoc', 'value')
+                                     'asoc': relationship('custom_column_' + str(row.id), uselist=False),
+                                     'value': association_proxy('asoc', 'value')
                                      }
                         books_custom_column_links[row.id] = type(str('books_custom_column_' + str(row.id) + '_link'),
                                                                  (Base,), dicttable)
@@ -462,7 +530,7 @@ class CalibreDB(threading.Thread):
                             'custom_column_' + str(cc_id[0]),
                             relationship(cc_classes[cc_id[0]],
                                          primaryjoin=(
-                                         Books.id == cc_classes[cc_id[0]].book),
+                                             Books.id == cc_classes[cc_id[0]].book),
                                          backref='books'))
                 elif (cc_id[1] == 'series'):
                     setattr(Books,
@@ -476,24 +544,27 @@ class CalibreDB(threading.Thread):
                                          secondary=books_custom_column_links[cc_id[0]],
                                          backref='books'))
 
-        Session = scoped_session(sessionmaker(autocommit=False,
-                                              autoflush=False,
-                                              bind=self.engine))
-        self.session = Session()
+        cls.session_factory = scoped_session(sessionmaker(autocommit=False,
+                                                          autoflush=True,
+                                                          bind=cls.engine))
+        for inst in cls.instances:
+            inst.initSession()
+
+        cls._init = True
         return True
 
     def get_book(self, book_id):
         return self.session.query(Books).filter(Books.id == book_id).first()
 
     def get_filtered_book(self, book_id, allow_show_archived=False):
-        return self.session.query(Books).filter(Books.id == book_id).\
+        return self.session.query(Books).filter(Books.id == book_id). \
             filter(self.common_filters(allow_show_archived)).first()
 
     def get_book_by_uuid(self, book_uuid):
         return self.session.query(Books).filter(Books.uuid == book_uuid).first()
 
-    def get_book_format(self, book_id, format):
-        return self.session.query(Data).filter(Data.book == book_id).filter(Data.format == format).first()
+    def get_book_format(self, book_id, file_format):
+        return self.session.query(Data).filter(Data.book == book_id).filter(Data.format == file_format).first()
 
     # Language and content filters for displaying in the UI
     def common_filters(self, allow_show_archived=False):
@@ -533,10 +604,12 @@ class CalibreDB(threading.Thread):
                     pos_content_cc_filter, ~neg_content_cc_filter, archived_filter)
 
     # Fill indexpage with all requested data from database
-    def fill_indexpage(self, page, database, db_filter, order, *join):
-        return self.fill_indexpage_with_archived_books(page, database, db_filter, order, False, *join)
+    def fill_indexpage(self, page, pagesize, database, db_filter, order, *join):
+        return self.fill_indexpage_with_archived_books(page, pagesize, database, db_filter, order, False, *join)
 
-    def fill_indexpage_with_archived_books(self, page, database, db_filter, order, allow_show_archived, *join):
+    def fill_indexpage_with_archived_books(self, page, pagesize, database, db_filter, order, allow_show_archived,
+                                           *join):
+        pagesize = pagesize or self.config.config_books_per_page
         if current_user.show_detail_random():
             randm = self.session.query(Books) \
                 .filter(self.common_filters(allow_show_archived)) \
@@ -544,16 +617,21 @@ class CalibreDB(threading.Thread):
                 .limit(self.config.config_random_books)
         else:
             randm = false()
-        off = int(int(self.config.config_books_per_page) * (page - 1))
+        off = int(int(pagesize) * (page - 1))
         query = self.session.query(database) \
             .join(*join, isouter=True) \
             .filter(db_filter) \
             .filter(self.common_filters(allow_show_archived))
-        pagination = Pagination(page, self.config.config_books_per_page,
-                                len(query.all()))
-        entries = query.order_by(*order).offset(off).limit(self.config.config_books_per_page).all()
-        for book in entries:
-            book = self.order_authors(book)
+        entries = list()
+        pagination = list()
+        try:
+            pagination = Pagination(page, pagesize,
+                                    len(query.all()))
+            entries = query.order_by(*order).offset(off).limit(pagesize).all()
+        except Exception as e:
+            log.debug_or_exception(e)
+        #for book in entries:
+        #    book = self.order_authors(book)
         return entries, randm, pagination
 
     # Orders all Authors in the list according to authors sort
@@ -561,13 +639,16 @@ class CalibreDB(threading.Thread):
         sort_authors = entry.author_sort.split('&')
         authors_ordered = list()
         error = False
+        ids = [a.id for a in entry.authors]
         for auth in sort_authors:
+            results = self.session.query(Authors).filter(Authors.sort == auth.lstrip().strip()).all()
             # ToDo: How to handle not found authorname
-            result = self.session.query(Authors).filter(Authors.sort == auth.lstrip().strip()).first()
-            if not result:
+            if not len(results):
                 error = True
                 break
-            authors_ordered.append(result)
+            for r in results:
+                if r.id in ids:
+                    authors_ordered.append(r)
         if not error:
             entry.authors = authors_ordered
         return entry
@@ -587,24 +668,37 @@ class CalibreDB(threading.Thread):
         for authorterm in authorterms:
             q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + authorterm + "%")))
 
-        return self.session.query(Books)\
+        return self.session.query(Books) \
             .filter(and_(Books.authors.any(and_(*q)), func.lower(Books.title).ilike("%" + title + "%"))).first()
 
     # read search results from calibre-database and return it (function is used for feed and simple search
-    def get_search_results(self, term):
+    def get_search_results(self, term, offset=None, order=None, limit=None):
+        order = order or [Books.sort]
+        pagination = None
         term.strip().lower()
         self.session.connection().connection.connection.create_function("lower", 1, lcase)
         q = list()
         authorterms = re.split("[, ]+", term)
         for authorterm in authorterms:
             q.append(Books.authors.any(func.lower(Authors.name).ilike("%" + authorterm + "%")))
-        return self.session.query(Books).filter(self.common_filters(True)).filter(
+        result = self.session.query(Books).filter(self.common_filters(True)).filter(
             or_(Books.tags.any(func.lower(Tags.name).ilike("%" + term + "%")),
                 Books.series.any(func.lower(Series.name).ilike("%" + term + "%")),
                 Books.authors.any(and_(*q)),
                 Books.publishers.any(func.lower(Publishers.name).ilike("%" + term + "%")),
                 func.lower(Books.title).ilike("%" + term + "%")
-                )).order_by(Books.sort).all()
+                )).order_by(*order).all()
+        result_count = len(result)
+        if offset != None and limit != None:
+            offset = int(offset)
+            limit_all = offset + int(limit)
+            pagination = Pagination((offset / (int(limit)) + 1), limit, result_count)
+        else:
+            offset = 0
+            limit_all = result_count
+
+        ub.store_ids(result)
+        return result[offset:limit_all], result_count, pagination
 
     # Creates for all stored languages a translated speaking name in the array for the UI
     def speaking_language(self, languages=None):
@@ -638,17 +732,23 @@ class CalibreDB(threading.Thread):
         conn = conn or self.session.connection().connection.connection
         conn.create_function("title_sort", 1, _title_sort)
 
-    def dispose(self):
+    @classmethod
+    def dispose(cls):
         # global session
 
-        old_session = self.session
-        self.session = None
-        if old_session:
-            try: old_session.close()
-            except: pass
-            if old_session.bind:
-                try: old_session.bind.dispose()
-                except Exception: pass
+        for inst in cls.instances:
+            old_session = inst.session
+            inst.session = None
+            if old_session:
+                try:
+                    old_session.close()
+                except Exception:
+                    pass
+                if old_session.bind:
+                    try:
+                        old_session.bind.dispose()
+                    except Exception:
+                        pass
 
         for attr in list(Books.__dict__.keys()):
             if attr.startswith("custom_column_"):
@@ -665,14 +765,15 @@ class CalibreDB(threading.Thread):
                     Base.metadata.remove(table)
 
     def reconnect_db(self, config, app_db_path):
-        self.session.close()
+        self.dispose()
         self.engine.dispose()
         self.setup_db(config, app_db_path)
 
+
 def lcase(s):
     try:
         return unidecode.unidecode(s.lower())
     except Exception as e:
         log = logger.create()
-        log.exception(e)
+        log.debug_or_exception(e)
         return s.lower()

cps/debug_info.py

@@ -0,0 +1,65 @@
+# -*- coding: utf-8 -*-
+
+#  This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#    Copyright (C) 2012-2019 cervinko, idalin, SiphonSquirrel, ouzklcn, akushsky,
+#                            OzzieIsaacs, bodybybuddha, jkrehm, matthazinski, janeczku
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+import shutil
+import glob
+import zipfile
+import json
+from io import BytesIO
+try:
+    from StringIO import StringIO
+except ImportError:
+    from io import StringIO
+
+import os
+
+from flask import send_file
+
+from . import logger, config
+from .about import collect_stats
+
+log = logger.create()
+
+def assemble_logfiles(file_name):
+    log_list = sorted(glob.glob(file_name + '*'), reverse=True)
+    wfd = StringIO()
+    for f in log_list:
+        with open(f, 'r') as fd:
+            shutil.copyfileobj(fd, wfd)
+    wfd.seek(0)
+    return send_file(wfd,
+                     as_attachment=True,
+                     attachment_filename=os.path.basename(file_name))
+
+def send_debug():
+    file_list = glob.glob(logger.get_logfile(config.config_logfile) + '*')
+    file_list.extend(glob.glob(logger.get_accesslogfile(config.config_access_logfile) + '*'))
+    for element in [logger.LOG_TO_STDOUT, logger.LOG_TO_STDERR]:
+        if element in file_list:
+            file_list.remove(element)
+    memory_zip = BytesIO()
+    with zipfile.ZipFile(memory_zip, 'w', compression=zipfile.ZIP_DEFLATED) as zf:
+        zf.writestr('settings.txt', json.dumps(config.toDict()))
+        zf.writestr('libs.txt', json.dumps(collect_stats()))
+        for fp in file_list:
+            zf.write(fp, os.path.basename(fp))
+    memory_zip.seek(0)
+    return send_file(memory_zip,
+                     as_attachment=True,
+                     attachment_filename="Calibre-Web-debug-pack.zip")

cps/editbooks.py

@@ -30,18 +30,45 @@ from uuid import uuid4
 from flask import Blueprint, request, flash, redirect, url_for, abort, Markup, Response
 from flask_babel import gettext as _
 from flask_login import current_user, login_required
-from sqlalchemy.exc import OperationalError
-
+from sqlalchemy.exc import OperationalError, IntegrityError
+from sqlite3 import OperationalError as sqliteOperationalError
 from . import constants, logger, isoLanguages, gdriveutils, uploader, helper
-from . import config, get_locale, ub, worker, db
+from . import config, get_locale, ub, db
 from . import calibre_db
-from .web import login_required_if_no_ano, render_title_template, edit_required, upload_required
+from .services.worker import WorkerThread
+from .tasks.upload import TaskUpload
+from .render_template import render_title_template
+from .usermanagement import login_required_if_no_ano
+
+try:
+    from functools import wraps
+except ImportError:
+    pass  # We're not using Python 3
 
 
 editbook = Blueprint('editbook', __name__)
 log = logger.create()
 
 
+def upload_required(f):
+    @wraps(f)
+    def inner(*args, **kwargs):
+        if current_user.role_upload() or current_user.role_admin():
+            return f(*args, **kwargs)
+        abort(403)
+
+    return inner
+
+def edit_required(f):
+    @wraps(f)
+    def inner(*args, **kwargs):
+        if current_user.role_edit() or current_user.role_admin():
+            return f(*args, **kwargs)
+        abort(403)
+
+    return inner
+
+
 # Modifies different Database objects, first check if elements have to be added to database, than check
 # if elements have to be deleted, because they are no longer used
 def modify_database_object(input_elements, db_book_object, db_object, db_session, db_type):
@@ -172,27 +199,48 @@ def modify_identifiers(input_identifiers, db_identifiers, db_session):
             changed = True
     return changed, error
 
-
-@editbook.route("/delete/<int:book_id>/", defaults={'book_format': ""})
-@editbook.route("/delete/<int:book_id>/<string:book_format>/")
+@editbook.route("/ajax/delete/<int:book_id>")
 @login_required
-def delete_book(book_id, book_format):
+def delete_book_from_details(book_id):
+    return Response(delete_book(book_id,"", True), mimetype='application/json')
+
+
+@editbook.route("/delete/<int:book_id>", defaults={'book_format': ""})
+@editbook.route("/delete/<int:book_id>/<string:book_format>")
+@login_required
+def delete_book_ajax(book_id, book_format):
+    return delete_book(book_id,book_format, False)
+
+def delete_book(book_id, book_format, jsonResponse):
+    warning = {}
     if current_user.role_delete_books():
         book = calibre_db.get_book(book_id)
         if book:
             try:
                 result, error = helper.delete_book(book, config.config_calibre_dir, book_format=book_format.upper())
                 if not result:
-                    flash(error, category="error")
-                    return redirect(url_for('editbook.edit_book', book_id=book_id))
+                    if jsonResponse:
+                        return json.dumps({"location": url_for("editbook.edit_book"),
+                                           "type": "alert",
+                                           "format": "",
+                                           "error": error}),
+                    else:
+                        flash(error, category="error")
+                        return redirect(url_for('editbook.edit_book', book_id=book_id))
                 if error:
-                    flash(error, category="warning")
+                    if jsonResponse:
+                        warning = {"location": url_for("editbook.edit_book"),
+                                                "type": "warning",
+                                                "format": "",
+                                                "error": error}
+                    else:
+                        flash(error, category="warning")
                 if not book_format:
                     # delete book from Shelfs, Downloads, Read list
                     ub.session.query(ub.BookShelf).filter(ub.BookShelf.book_id == book_id).delete()
                     ub.session.query(ub.ReadBook).filter(ub.ReadBook.book_id == book_id).delete()
                     ub.delete_download(book_id)
-                    ub.session.commit()
+                    ub.session_commit()
 
                     # check if only this book links to:
                     # author, language, series, tags, custom columns
@@ -236,23 +284,34 @@ def delete_book(book_id, book_format):
                         filter(db.Data.format == book_format).delete()
                 calibre_db.session.commit()
             except Exception as e:
-                log.debug(e)
+                log.debug_or_exception(e)
                 calibre_db.session.rollback()
         else:
             # book not found
             log.error('Book with id "%s" could not be deleted: not found', book_id)
     if book_format:
-        flash(_('Book Format Successfully Deleted'), category="success")
-        return redirect(url_for('editbook.edit_book', book_id=book_id))
+        if jsonResponse:
+            return json.dumps([warning, {"location": url_for("editbook.edit_book", book_id=book_id),
+                                         "type": "success",
+                                         "format": book_format,
+                                         "message": _('Book Format Successfully Deleted')}])
+        else:
+            flash(_('Book Format Successfully Deleted'), category="success")
+            return redirect(url_for('editbook.edit_book', book_id=book_id))
     else:
-        flash(_('Book Successfully Deleted'), category="success")
-        return redirect(url_for('web.index'))
+        if jsonResponse:
+            return json.dumps([warning, {"location": url_for('web.index'),
+                                         "type": "success",
+                                         "format": book_format,
+                                         "message": _('Book Successfully Deleted')}])
+        else:
+            flash(_('Book Successfully Deleted'), category="success")
+            return redirect(url_for('web.index'))
 
 
 def render_edit_book(book_id):
-    calibre_db.update_title_sort(config)
     cc = calibre_db.session.query(db.Custom_Columns).filter(db.Custom_Columns.datatype.notin_(db.cc_exceptions)).all()
-    book = calibre_db.get_filtered_book(book_id)
+    book = calibre_db.get_filtered_book(book_id, allow_show_archived=True)
     if not book:
         flash(_(u"Error opening eBook. File does not exist or file is not accessible"), category="error")
         return redirect(url_for("web.index"))
@@ -272,7 +331,7 @@ def render_edit_book(book_id):
     kepub_possible=None
     if config.config_converterpath:
         for file in book.data:
-            if file.format.lower() in constants.EXTENSIONS_CONVERT:
+            if file.format.lower() in constants.EXTENSIONS_CONVERT_FROM:
                 valid_source_formats.append(file.format.lower())
     if config.config_kepubifypath and 'epub' in [file.format.lower() for file in book.data]:
         kepub_possible = True
@@ -281,7 +340,7 @@ def render_edit_book(book_id):
 
     # Determine what formats don't already exist
     if config.config_converterpath:
-        allowed_conversion_formats = constants.EXTENSIONS_CONVERT[:]
+        allowed_conversion_formats = constants.EXTENSIONS_CONVERT_TO[:]
         for file in book.data:
             if file.format.lower() in allowed_conversion_formats:
                 allowed_conversion_formats.remove(file.format.lower())
@@ -357,7 +416,10 @@ def edit_book_comments(comments, book):
 def edit_book_languages(languages, book, upload=False):
     input_languages = languages.split(',')
     unknown_languages = []
-    input_l = isoLanguages.get_language_codes(get_locale(), input_languages, unknown_languages)
+    if not upload:
+        input_l = isoLanguages.get_language_codes(get_locale(), input_languages, unknown_languages)
+    else:
+        input_l = isoLanguages.get_valid_language_codes(get_locale(), input_languages, unknown_languages)
     for l in unknown_languages:
         log.error('%s is not a valid language', l)
         flash(_(u"%(langname)s is not a valid language", langname=l), category="warning")
@@ -367,7 +429,7 @@ def edit_book_languages(languages, book, upload=False):
         # the book it's language is set to the filter language
         if input_l[0] != current_user.filter_language() and current_user.filter_language() != "all":
             input_l[0] = calibre_db.session.query(db.Languages). \
-                filter(db.Languages.lang_code == current_user.filter_language()).first()
+                filter(db.Languages.lang_code == current_user.filter_language()).first().lang_code
     # Remove duplicates
     input_l = helper.uniq(input_l)
     return modify_database_object(input_l, book.languages, db.Languages, calibre_db.session, 'languages')
@@ -466,9 +528,11 @@ def upload_single_file(request, book, book_id):
         requested_file = request.files['btn-upload-format']
         # check for empty request
         if requested_file.filename != '':
+            if not current_user.role_upload():
+                abort(403)
             if '.' in requested_file.filename:
                 file_ext = requested_file.filename.rsplit('.', 1)[-1].lower()
-                if file_ext not in constants.EXTENSIONS_UPLOAD:
+                if file_ext not in constants.EXTENSIONS_UPLOAD and '' not in constants.EXTENSIONS_UPLOAD:
                     flash(_("File extension '%(ext)s' is not allowed to be uploaded to this server", ext=file_ext),
                           category="error")
                     return redirect(url_for('web.show_book', book_id=book.id))
@@ -505,7 +569,7 @@ def upload_single_file(request, book, book_id):
                     calibre_db.session.add(db_format)
                     calibre_db.session.commit()
                     calibre_db.update_title_sort(config)
-                except OperationalError as e:
+                except (OperationalError, IntegrityError) as e:
                     calibre_db.session.rollback()
                     log.error('Database error: %s', e)
                     flash(_(u"Database error: %(error)s.", error=e), category="error")
@@ -513,8 +577,8 @@ def upload_single_file(request, book, book_id):
 
             # Queue uploader info
             uploadText=_(u"File format %(ext)s added to %(book)s", ext=file_ext.upper(), book=book.title)
-            worker.add_upload(current_user.nickname,
-                "<a href=\"" + url_for('web.show_book', book_id=book.id) + "\">" + uploadText + "</a>")
+            WorkerThread.add(current_user.nickname, TaskUpload(
+                "<a href=\"" + url_for('web.show_book', book_id=book.id) + "\">" + uploadText + "</a>"))
 
             return uploader.process(
                 saved_filename, *os.path.splitext(requested_file.filename),
@@ -526,6 +590,8 @@ def upload_cover(request, book):
         requested_file = request.files['btn-upload-cover']
         # check for empty request
         if requested_file.filename != '':
+            if not current_user.role_upload():
+                abort(403)
             ret, message = helper.save_cover(requested_file, book.path)
             if ret is True:
                 return True
@@ -540,13 +606,20 @@ def upload_cover(request, book):
 @edit_required
 def edit_book(book_id):
     modif_date = False
+
+    # create the function for sorting...
+    try:
+        calibre_db.update_title_sort(config)
+    except sqliteOperationalError as e:
+        log.debug_or_exception(e)
+        calibre_db.session.rollback()
+
     # Show form
     if request.method != 'POST':
         return render_edit_book(book_id)
 
-    # create the function for sorting...
-    calibre_db.update_title_sort(config)
-    book = calibre_db.get_filtered_book(book_id)
+
+    book = calibre_db.get_filtered_book(book_id, allow_show_archived=True)
 
     # Book not found
     if not book:
@@ -562,6 +635,7 @@ def edit_book(book_id):
         merge_metadata(to_save, meta)
         # Update book
         edited_books_id = None
+
         #handle book title
         if book.title != to_save["book_title"].rstrip().strip():
             if to_save["book_title"] == '':
@@ -605,13 +679,19 @@ def edit_book(book_id):
             error = helper.update_dir_stucture(edited_books_id, config.config_calibre_dir, input_authors[0])
 
         if not error:
-            if to_save["cover_url"]:
-                result, error = helper.save_cover_from_url(to_save["cover_url"], book.path)
-                if result is True:
-                    book.has_cover = 1
-                    modif_date = True
-                else:
-                    flash(error, category="error")
+            if "cover_url" in to_save:
+                if to_save["cover_url"]:
+                    if not current_user.role_upload():
+                        return "", (403)
+                    if to_save["cover_url"].endswith('/static/generic_cover.jpg'):
+                        book.has_cover = 0
+                    else:
+                        result, error = helper.save_cover_from_url(to_save["cover_url"], book.path)
+                        if result is True:
+                            book.has_cover = 1
+                            modif_date = True
+                        else:
+                            flash(error, category="error")
 
             # Add default series_index to book
             modif_date |= edit_book_series_index(to_save["series_index"], book)
@@ -653,6 +733,7 @@ def edit_book(book_id):
 
             if modif_date:
                 book.last_modified = datetime.utcnow()
+            calibre_db.session.merge(book)
             calibre_db.session.commit()
             if config.config_use_google_drive:
                 gdriveutils.updateGdriveCalibreFromLocal()
@@ -666,7 +747,7 @@ def edit_book(book_id):
             flash(error, category="error")
             return render_edit_book(book_id)
     except Exception as e:
-        log.exception(e)
+        log.debug_or_exception(e)
         calibre_db.session.rollback()
         flash(_("Error editing book, please check logfile for details"), category="error")
         return redirect(url_for('web.show_book', book_id=book.id))
@@ -716,7 +797,7 @@ def upload():
                 # check if file extension is correct
                 if '.' in requested_file.filename:
                     file_ext = requested_file.filename.rsplit('.', 1)[-1].lower()
-                    if file_ext not in constants.EXTENSIONS_UPLOAD:
+                    if file_ext not in constants.EXTENSIONS_UPLOAD and '' not in constants.EXTENSIONS_UPLOAD:
                         flash(
                             _("File extension '%(ext)s' is not allowed to be uploaded to this server",
                               ext=file_ext), category="error")
@@ -768,42 +849,17 @@ def upload():
                         if not db_author:
                             db_author = stored_author
                         sort_author = stored_author.sort
-                    sort_authors_list.append(sort_author) # helper.get_sorted_author(sort_author))
+                    sort_authors_list.append(sort_author)
                 sort_authors = ' & '.join(sort_authors_list)
 
                 title_dir = helper.get_valid_filename(title)
                 author_dir = helper.get_valid_filename(db_author.name)
-                filepath = os.path.join(config.config_calibre_dir, author_dir, title_dir)
-                saved_filename = os.path.join(filepath, title_dir + meta.extension.lower())
-
-                # check if file path exists, otherwise create it, copy file to calibre path and delete temp file
-                if not os.path.exists(filepath):
-                    try:
-                        os.makedirs(filepath)
-                    except OSError:
-                        log.error("Failed to create path %s (Permission denied)", filepath)
-                        flash(_(u"Failed to create path %(path)s (Permission denied).", path=filepath), category="error")
-                        return Response(json.dumps({"location": url_for("web.index")}), mimetype='application/json')
-                try:
-                    copyfile(meta.file_path, saved_filename)
-                    os.unlink(meta.file_path)
-                except OSError as e:
-                    log.error("Failed to move file %s: %s", saved_filename, e)
-                    flash(_(u"Failed to Move File %(file)s: %(error)s", file=saved_filename, error=e), category="error")
-                    return Response(json.dumps({"location": url_for("web.index")}), mimetype='application/json')
-
-                if meta.cover is None:
-                    has_cover = 0
-                    copyfile(os.path.join(constants.STATIC_DIR, 'generic_cover.jpg'),
-                             os.path.join(filepath, "cover.jpg"))
-                else:
-                    has_cover = 1
 
                 # combine path and normalize path from windows systems
                 path = os.path.join(author_dir, title_dir).replace('\\', '/')
                 # Calibre adds books with utc as timezone
                 db_book = db.Books(title, "", sort_authors, datetime.utcnow(), datetime(101, 1, 1),
-                                   '1', datetime.utcnow(), path, has_cover, db_author, [], "")
+                                   '1', datetime.utcnow(), path, meta.cover, db_author, [], "")
 
                 modif_date |= modify_database_object(input_authors, db_book.authors, db.Authors, calibre_db.session,
                                                      'author')
@@ -821,7 +877,7 @@ def upload():
                 modif_date |= edit_book_series(meta.series, db_book)
 
                 # Add file to book
-                file_size = os.path.getsize(saved_filename)
+                file_size = os.path.getsize(meta.file_path)
                 db_data = db.Data(db_book, meta.extension.upper()[1:], file_size, title_dir)
                 db_book.data.append(db_data)
                 calibre_db.session.add(db_book)
@@ -829,39 +885,44 @@ def upload():
                 # flush content, get db_book.id available
                 calibre_db.session.flush()
 
-                # Comments needs book id therfore only possiblw after flush
+                # Comments needs book id therfore only possible after flush
                 modif_date |= edit_book_comments(Markup(meta.description).unescape(), db_book)
 
                 book_id = db_book.id
                 title = db_book.title
 
-                error = helper.update_dir_stucture(book_id, config.config_calibre_dir, input_authors[0])
+                error = helper.update_dir_structure_file(book_id,
+                                                   config.config_calibre_dir,
+                                                   input_authors[0],
+                                                   meta.file_path,
+                                                   title_dir + meta.extension)
 
                 # move cover to final directory, including book id
-                if has_cover:
-                    new_coverpath = os.path.join(config.config_calibre_dir, db_book.path, "cover.jpg")
-                    try:
-                        copyfile(meta.cover, new_coverpath)
+                if meta.cover:
+                    coverfile = meta.cover
+                else:
+                    coverfile = os.path.join(constants.STATIC_DIR, 'generic_cover.jpg')
+                new_coverpath = os.path.join(config.config_calibre_dir, db_book.path, "cover.jpg")
+                try:
+                    copyfile(coverfile, new_coverpath)
+                    if meta.cover:
                         os.unlink(meta.cover)
-                    except OSError as e:
-                        log.error("Failed to move cover file %s: %s", new_coverpath, e)
-                        flash(_(u"Failed to Move Cover File %(file)s: %(error)s", file=new_coverpath,
-                                error=e),
-                              category="error")
+                except OSError as e:
+                    log.error("Failed to move cover file %s: %s", new_coverpath, e)
+                    flash(_(u"Failed to Move Cover File %(file)s: %(error)s", file=new_coverpath,
+                            error=e),
+                          category="error")
 
                 # save data to database, reread data
                 calibre_db.session.commit()
-                #calibre_db.setup_db(config, ub.app_DB_path)
-                # Reread book. It's important not to filter the result, as it could have language which hide it from
-                # current users view (tags are not stored/extracted from metadata and could also be limited)
-                #book = calibre_db.get_book(book_id)
+
                 if config.config_use_google_drive:
                     gdriveutils.updateGdriveCalibreFromLocal()
                 if error:
                     flash(error, category="error")
                 uploadText=_(u"File %(file)s uploaded", file=title)
-                worker.add_upload(current_user.nickname,
-                    "<a href=\"" + url_for('web.show_book', book_id=book_id) + "\">" + uploadText + "</a>")
+                WorkerThread.add(current_user.nickname, TaskUpload(
+                    "<a href=\"" + url_for('web.show_book', book_id=book_id) + "\">" + uploadText + "</a>"))
 
                 if len(request.files.getlist("btn-upload")) < 2:
                     if current_user.role_edit() or current_user.role_admin():
@@ -870,7 +931,7 @@ def upload():
                     else:
                         resp = {"location": url_for('web.show_book', book_id=book_id)}
                         return Response(json.dumps(resp), mimetype='application/json')
-            except OperationalError as e:
+            except (OperationalError, IntegrityError) as e:
                 calibre_db.session.rollback()
                 log.error("Database error: %s", e)
                 flash(_(u"Database error: %(error)s.", error=e), category="error")
@@ -899,3 +960,113 @@ def convert_bookformat(book_id):
     else:
         flash(_(u"There was an error converting this book: %(res)s", res=rtn), category="error")
     return redirect(url_for('editbook.edit_book', book_id=book_id))
+
+@editbook.route("/ajax/editbooks/<param>", methods=['POST'])
+@login_required_if_no_ano
+@edit_required
+def edit_list_book(param):
+    vals = request.form.to_dict()
+    book = calibre_db.get_book(vals['pk'])
+    if param =='series_index':
+        edit_book_series_index(vals['value'], book)
+    elif param =='tags':
+        edit_book_tags(vals['value'], book)
+    elif param =='series':
+        edit_book_series(vals['value'], book)
+    elif param =='publishers':
+        vals['publisher'] = vals['value']
+        edit_book_publisher(vals, book)
+    elif param =='languages':
+        edit_book_languages(vals['value'], book)
+    elif param =='author_sort':
+        book.author_sort = vals['value']
+    elif param =='title':
+        book.title = vals['value']
+        helper.update_dir_stucture(book.id, config.config_calibre_dir)
+    elif param =='sort':
+        book.sort = vals['value']
+    # ToDo: edit books
+    elif param =='authors':
+        input_authors = vals['value'].split('&')
+        input_authors = list(map(lambda it: it.strip().replace(',', '|'), input_authors))
+        modify_database_object(input_authors, book.authors, db.Authors, calibre_db.session, 'author')
+        sort_authors_list = list()
+        for inp in input_authors:
+            stored_author = calibre_db.session.query(db.Authors).filter(db.Authors.name == inp).first()
+            if not stored_author:
+                stored_author = helper.get_sorted_author(inp)
+            else:
+                stored_author = stored_author.sort
+            sort_authors_list.append(helper.get_sorted_author(stored_author))
+        sort_authors = ' & '.join(sort_authors_list)
+        if book.author_sort != sort_authors:
+            book.author_sort = sort_authors
+        helper.update_dir_stucture(book.id, config.config_calibre_dir, input_authors[0])
+    book.last_modified = datetime.utcnow()
+    calibre_db.session.commit()
+    return ""
+
+@editbook.route("/ajax/sort_value/<field>/<int:bookid>")
+@login_required
+def get_sorted_entry(field, bookid):
+    if field == 'title' or field == 'authors':
+        book = calibre_db.get_filtered_book(bookid)
+        if book:
+            if field == 'title':
+                return json.dumps({'sort': book.sort})
+            elif field == 'authors':
+                return json.dumps({'author_sort': book.author_sort})
+    return ""
+
+
+@editbook.route("/ajax/simulatemerge", methods=['POST'])
+@login_required
+@edit_required
+def simulate_merge_list_book():
+    vals = request.get_json().get('Merge_books')
+    if vals:
+        to_book = calibre_db.get_book(vals[0]).title
+        vals.pop(0)
+        if to_book:
+            for book_id in vals:
+                from_book = []
+                from_book.append(calibre_db.get_book(book_id).title)
+            return json.dumps({'to': to_book, 'from': from_book})
+    return ""
+
+
+@editbook.route("/ajax/mergebooks", methods=['POST'])
+@login_required
+@edit_required
+def merge_list_book():
+    vals = request.get_json().get('Merge_books')
+    to_file = list()
+    if vals:
+        # load all formats from target book
+        to_book = calibre_db.get_book(vals[0])
+        vals.pop(0)
+        if to_book:
+            for file in to_book.data:
+                to_file.append(file.format)
+            to_name = helper.get_valid_filename(to_book.title) + ' - ' + \
+                      helper.get_valid_filename(to_book.authors[0].name)
+            for book_id in vals:
+                from_book = calibre_db.get_book(book_id)
+                if from_book:
+                    for element in from_book.data:
+                        if element.format not in to_file:
+                            # create new data entry with: book_id, book_format, uncompressed_size, name
+                            filepath_new = os.path.normpath(os.path.join(config.config_calibre_dir,
+                                                                         to_book.path,
+                                                                         to_name + "." + element.format.lower()))
+                            filepath_old = os.path.normpath(os.path.join(config.config_calibre_dir,
+                                                                         from_book.path,
+                                                                         element.name + "." + element.format.lower()))
+                            copyfile(filepath_old, filepath_new)
+                            to_book.data.append(db.Data(to_book.id,
+                                                        element.format,
+                                                        element.uncompressed_size,
+                                                        to_name))
+                    delete_book(from_book.id,"", True) # json_resp =
+                    return json.dumps({'success': True})
+    return ""

cps/epub.py

@@ -26,6 +26,7 @@ from .helper import split_authors
 from .constants import BookMeta
 
 
+
 def extractCover(zipFile, coverFile, coverpath, tmp_file_name):
     if coverFile is None:
         return None
@@ -65,11 +66,11 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
         tmp = p.xpath('dc:%s/text()' % s, namespaces=ns)
         if len(tmp) > 0:
             if s == 'creator':
-                epub_metadata[s] = ' & '.join(split_authors(p.xpath('dc:%s/text()' % s, namespaces=ns)))
+                epub_metadata[s] = ' & '.join(split_authors(tmp))
             elif s == 'subject':
-                epub_metadata[s] = ', '.join(p.xpath('dc:%s/text()' % s, namespaces=ns))
+                epub_metadata[s] = ', '.join(tmp)
             else:
-                epub_metadata[s] = p.xpath('dc:%s/text()' % s, namespaces=ns)[0]
+                epub_metadata[s] = tmp[0]
         else:
             epub_metadata[s] = u'Unknown'
 
@@ -83,16 +84,8 @@ def get_epub_info(tmp_file_path, original_file_name, original_file_extension):
         else:
             epub_metadata['description'] = ""
 
-    if epub_metadata['language'] == u'Unknown':
-        epub_metadata['language'] = ""
-    else:
-        lang = epub_metadata['language'].split('-', 1)[0].lower()
-        if len(lang) == 2:
-            epub_metadata['language'] = isoLanguages.get(part1=lang).name
-        elif len(lang) == 3:
-            epub_metadata['language'] = isoLanguages.get(part3=lang).name
-        else:
-            epub_metadata['language'] = ""
+    lang = epub_metadata['language'].split('-', 1)[0].lower()
+    epub_metadata['language'] = isoLanguages.get_lang3(lang)
 
     series = tree.xpath("/pkg:package/pkg:metadata/pkg:meta[@name='calibre:series']/@content", namespaces=ns)
     if len(series) > 0:

cps/error_handler.py

@@ -0,0 +1,73 @@
+# -*- coding: utf-8 -*-
+
+#  This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#    Copyright (C) 2018-2020 OzzieIsaacs
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+import traceback
+from flask import render_template
+from werkzeug.exceptions import default_exceptions
+try:
+    from werkzeug.exceptions import FailedDependency
+except ImportError:
+    from werkzeug.exceptions import UnprocessableEntity as FailedDependency
+
+from . import config, app, logger, services
+
+
+log = logger.create()
+
+# custom error page
+def error_http(error):
+    return render_template('http_error.html',
+                           error_code="Error {0}".format(error.code),
+                           error_name=error.name,
+                           issue=False,
+                           instance=config.config_calibre_web_title
+                           ), error.code
+
+
+def internal_error(error):
+    return render_template('http_error.html',
+                           error_code="Internal Server Error",
+                           error_name=str(error),
+                           issue=True,
+                           error_stack=traceback.format_exc().split("\n"),
+                           instance=config.config_calibre_web_title
+                           ), 500
+
+def init_errorhandler():
+    # http error handling
+    for ex in default_exceptions:
+        if ex < 500:
+            app.register_error_handler(ex, error_http)
+        elif ex == 500:
+            app.register_error_handler(ex, internal_error)
+
+
+    if services.ldap:
+        # Only way of catching the LDAPException upon logging in with LDAP server down
+        @app.errorhandler(services.ldap.LDAPException)
+        def handle_exception(e):
+            log.debug('LDAP server not accessible while trying to login to opds feed')
+            return error_http(FailedDependency())
+
+
+# @app.errorhandler(InvalidRequestError)
+#@app.errorhandler(OperationalError)
+#def handle_db_exception(e):
+#    db.session.rollback()
+#    log.error('Database request error: %s',e)
+#    return internal_error(InternalServerError(e))

cps/gdrive.py

@@ -35,22 +35,22 @@ from flask_babel import gettext as _
 from flask_login import login_required
 
 from . import logger, gdriveutils, config, ub, calibre_db
-from .web import admin_required
+from .admin import admin_required
 
-gdrive = Blueprint('gdrive', __name__)
+gdrive = Blueprint('gdrive', __name__, url_prefix='/gdrive')
 log = logger.create()
 
 try:
     from googleapiclient.errors import HttpError
 except ImportError as err:
-    log.debug(("Cannot import googleapiclient, using gdrive will not work: %s", err))
+    log.debug("Cannot import googleapiclient, using GDrive will not work: %s", err)
 
 current_milli_time = lambda: int(round(time() * 1000))
 
-gdrive_watch_callback_token = 'target=calibreweb-watch_files'
+gdrive_watch_callback_token = 'target=calibreweb-watch_files'  #nosec
 
 
-@gdrive.route("/gdrive/authenticate")
+@gdrive.route("/authenticate")
 @login_required
 @admin_required
 def authenticate_google_drive():
@@ -63,7 +63,7 @@ def authenticate_google_drive():
     return redirect(authUrl)
 
 
-@gdrive.route("/gdrive/callback")
+@gdrive.route("/callback")
 def google_drive_callback():
     auth_code = request.args.get('code')
     if not auth_code:
@@ -77,18 +77,14 @@ def google_drive_callback():
     return redirect(url_for('admin.configuration'))
 
 
-@gdrive.route("/gdrive/watch/subscribe")
+@gdrive.route("/watch/subscribe")
 @login_required
 @admin_required
 def watch_gdrive():
     if not config.config_google_drive_watch_changes_response:
         with open(gdriveutils.CLIENT_SECRETS, 'r') as settings:
             filedata = json.load(settings)
-        if filedata['web']['redirect_uris'][0].endswith('/'):
-            filedata['web']['redirect_uris'][0] = filedata['web']['redirect_uris'][0][:-((len('/gdrive/callback')+1))]
-        else:
-            filedata['web']['redirect_uris'][0] = filedata['web']['redirect_uris'][0][:-(len('/gdrive/callback'))]
-        address = '%s/gdrive/watch/callback' % filedata['web']['redirect_uris'][0]
+        address = filedata['web']['redirect_uris'][0].rstrip('/').replace('/gdrive/callback', '/gdrive/watch/callback')
         notification_id = str(uuid4())
         try:
             result = gdriveutils.watchChange(gdriveutils.Gdrive.Instance().drive, notification_id,
@@ -98,14 +94,15 @@ def watch_gdrive():
         except HttpError as e:
             reason=json.loads(e.content)['error']['errors'][0]
             if reason['reason'] == u'push.webhookUrlUnauthorized':
-                flash(_(u'Callback domain is not verified, please follow steps to verify domain in google developer console'), category="error")
+                flash(_(u'Callback domain is not verified, '
+                        u'please follow steps to verify domain in google developer console'), category="error")
             else:
                 flash(reason['message'], category="error")
 
     return redirect(url_for('admin.configuration'))
 
 
-@gdrive.route("/gdrive/watch/revoke")
+@gdrive.route("/watch/revoke")
 @login_required
 @admin_required
 def revoke_watch_gdrive():
@@ -121,40 +118,43 @@ def revoke_watch_gdrive():
     return redirect(url_for('admin.configuration'))
 
 
-@gdrive.route("/gdrive/watch/callback", methods=['GET', 'POST'])
+@gdrive.route("/watch/callback", methods=['GET', 'POST'])
 def on_received_watch_confirmation():
+    if not config.config_google_drive_watch_changes_response:
+        return ''
+    if request.headers.get('X-Goog-Channel-Token') != gdrive_watch_callback_token \
+            or request.headers.get('X-Goog-Resource-State') != 'change' \
+            or not request.data:
+        return ''
+
     log.debug('%r', request.headers)
-    if request.headers.get('X-Goog-Channel-Token') == gdrive_watch_callback_token \
-            and request.headers.get('X-Goog-Resource-State') == 'change' \
-            and request.data:
+    log.debug('%r', request.data)
+    log.info('Change received from gdrive')
 
-        data = request.data
+    try:
+        j = json.loads(request.data)
+        log.info('Getting change details')
+        response = gdriveutils.getChangeById(gdriveutils.Gdrive.Instance().drive, j['id'])
+        log.debug('%r', response)
+        if response:
+            if sys.version_info < (3, 0):
+                dbpath = os.path.join(config.config_calibre_dir, "metadata.db")
+            else:
+                dbpath = os.path.join(config.config_calibre_dir, "metadata.db").encode()
+            if not response['deleted'] and response['file']['title'] == 'metadata.db' \
+                and response['file']['md5Checksum'] != hashlib.md5(dbpath):
+                tmp_dir = os.path.join(tempfile.gettempdir(), 'calibre_web')
+                if not os.path.isdir(tmp_dir):
+                    os.mkdir(tmp_dir)
 
-        def updateMetaData():
-            log.info('Change received from gdrive')
-            log.debug('%r', data)
-            try:
-                j = json.loads(data)
-                log.info('Getting change details')
-                response = gdriveutils.getChangeById(gdriveutils.Gdrive.Instance().drive, j['id'])
-                log.debug('%r', response)
-                if response:
-                    if sys.version_info < (3, 0):
-                        dbpath = os.path.join(config.config_calibre_dir, "metadata.db")
-                    else:
-                        dbpath = os.path.join(config.config_calibre_dir, "metadata.db").encode()
-                    if not response['deleted'] and response['file']['title'] == 'metadata.db' \
-                       and response['file']['md5Checksum'] != hashlib.md5(dbpath):
-                        tmpDir = tempfile.gettempdir()
-                        log.info('Database file updated')
-                        copyfile(dbpath, os.path.join(tmpDir, "metadata.db_" + str(current_milli_time())))
-                        log.info('Backing up existing and downloading updated metadata.db')
-                        gdriveutils.downloadFile(None, "metadata.db", os.path.join(tmpDir, "tmp_metadata.db"))
-                        log.info('Setting up new DB')
-                        # prevent error on windows, as os.rename does on exisiting files
-                        move(os.path.join(tmpDir, "tmp_metadata.db"), dbpath)
-                        calibre_db.reconnect_db(config, ub.app_DB_path)
-            except Exception as e:
-                log.exception(e)
-        updateMetaData()
+                log.info('Database file updated')
+                copyfile(dbpath, os.path.join(tmp_dir, "metadata.db_" + str(current_milli_time())))
+                log.info('Backing up existing and downloading updated metadata.db')
+                gdriveutils.downloadFile(None, "metadata.db", os.path.join(tmp_dir, "tmp_metadata.db"))
+                log.info('Setting up new DB')
+                # prevent error on windows, as os.rename does on existing files, also allow cross hdd move
+                move(os.path.join(tmp_dir, "tmp_metadata.db"), dbpath)
+                calibre_db.reconnect_db(config, ub.app_DB_path)
+    except Exception as e:
+        log.debug_or_exception(e)
     return ''

cps/gdriveutils.py

@@ -20,6 +20,8 @@ from __future__ import division, print_function, unicode_literals
 import os
 import json
 import shutil
+import chardet
+import ssl
 
 from flask import Response, stream_with_context
 from sqlalchemy import create_engine
@@ -30,16 +32,25 @@ from sqlalchemy.ext.declarative import declarative_base
 from sqlalchemy.exc import OperationalError, InvalidRequestError
 
 try:
-    from pydrive.auth import GoogleAuth
-    from pydrive.drive import GoogleDrive
-    from pydrive.auth import RefreshError
     from apiclient import errors
     from httplib2 import ServerNotFoundError
-    gdrive_support = True
     importError = None
-except ImportError as err:
-    importError = err
+    gdrive_support = True
+except ImportError as e:
+    importError = e
     gdrive_support = False
+try:
+    from pydrive2.auth import GoogleAuth
+    from pydrive2.drive import GoogleDrive
+    from pydrive2.auth import RefreshError
+except ImportError as err:
+    try:
+        from pydrive.auth import GoogleAuth
+        from pydrive.drive import GoogleDrive
+        from pydrive.auth import RefreshError
+    except ImportError as err:
+        importError = err
+        gdrive_support = False
 
 from . import logger, cli, config
 from .constants import CONFIG_DIR as _CONFIG_DIR
@@ -89,7 +100,7 @@ class Singleton:
         except AttributeError:
             self._instance = self._decorated()
             return self._instance
-        except ImportError as e:
+        except (ImportError, NameError) as e:
             log.debug(e)
             return None
 
@@ -188,7 +199,7 @@ def getDrive(drive=None, gauth=None):
             except RefreshError as e:
                 log.error("Google Drive error: %s", e)
             except Exception as e:
-                log.exception(e)
+                log.debug_or_exception(e)
         else:
             # Initialize the saved creds
             gauth.Authorize()
@@ -206,14 +217,14 @@ def listRootFolders():
         drive = getDrive(Gdrive.Instance().drive)
         folder = "'root' in parents and mimeType = 'application/vnd.google-apps.folder' and trashed = false"
         fileList = drive.ListFile({'q': folder}).GetList()
-    except ServerNotFoundError as e:
+    except (ServerNotFoundError, ssl.SSLError) as e:
         log.info("GDrive Error %s" % e)
         fileList = []
     return fileList
 
 
 def getEbooksFolder(drive):
-    return getFolderInFolder('root',config.config_google_drive_folder,drive)
+    return getFolderInFolder('root', config.config_google_drive_folder, drive)
 
 
 def getFolderInFolder(parentId, folderName, drive):
@@ -243,7 +254,7 @@ def getEbooksFolderId(drive=None):
         gDriveId.path = '/'
         session.merge(gDriveId)
         session.commit()
-        return
+        return gDriveId.gdrive_id
 
 
 def getFile(pathId, fileName, drive):
@@ -383,7 +394,8 @@ def uploadFileToEbooksFolder(destFile, f):
             if len(existingFiles) > 0:
                 driveFile = existingFiles[0]
             else:
-                driveFile = drive.CreateFile({'title': x, 'parents': [{"kind": "drive#fileLink", 'id': parent['id']}],})
+                driveFile = drive.CreateFile({'title': x,
+                                              'parents': [{"kind": "drive#fileLink", 'id': parent['id']}], })
             driveFile.SetContentFile(f)
             driveFile.Upload()
         else:
@@ -545,21 +557,24 @@ def partial(total_byte_len, part_size_limit):
     return s
 
 # downloads files in chunks from gdrive
-def do_gdrive_download(df, headers):
+def do_gdrive_download(df, headers, convert_encoding=False):
     total_size = int(df.metadata.get('fileSize'))
     download_url = df.metadata.get('downloadUrl')
     s = partial(total_size, 1024 * 1024)  # I'm downloading BIG files, so 100M chunk size is fine for me
 
-    def stream():
+    def stream(convert_encoding):
         for byte in s:
             headers = {"Range": 'bytes=%s-%s' % (byte[0], byte[1])}
             resp, content = df.auth.Get_Http_Object().request(download_url, headers=headers)
             if resp.status == 206:
+                if convert_encoding:
+                    result = chardet.detect(content)
+                    content = content.decode(result['encoding']).encode('utf-8')
                 yield content
             else:
                 log.warning('An error occurred: %s', resp)
                 return
-    return Response(stream_with_context(stream()), headers=headers)
+    return Response(stream_with_context(stream(convert_encoding)), headers=headers)
 
 
 _SETTINGS_YAML_TEMPLATE = """

cps/helper.py

@@ -32,13 +32,12 @@ from tempfile import gettempdir
 import requests
 from babel.dates import format_datetime
 from babel.units import format_unit
-from flask import send_from_directory, make_response, redirect, abort
+from flask import send_from_directory, make_response, redirect, abort, url_for
 from flask_babel import gettext as _
 from flask_login import current_user
-from sqlalchemy.sql.expression import true, false, and_, text, func
+from sqlalchemy.sql.expression import true, false, and_, text
 from werkzeug.datastructures import Headers
 from werkzeug.security import generate_password_hash
-from . import calibre_db
 
 try:
     from urllib.parse import quote
@@ -51,141 +50,155 @@ try:
 except ImportError:
     use_unidecode = False
 
-try:
-    from PIL import Image as PILImage
-    from PIL import UnidentifiedImageError
-    use_PIL = True
-except ImportError:
-    use_PIL = False
-
-from . import logger, config, get_locale, db, ub, worker
+from . import calibre_db
+from .tasks.convert import TaskConvert
+from . import logger, config, get_locale, db, ub
 from . import gdriveutils as gd
 from .constants import STATIC_DIR as _STATIC_DIR
 from .subproc_wrapper import process_wait
-from .worker import STAT_WAITING, STAT_FAIL, STAT_STARTED, STAT_FINISH_SUCCESS
-from .worker import TASK_EMAIL, TASK_CONVERT, TASK_UPLOAD, TASK_CONVERT_ANY
-
+from .services.worker import WorkerThread, STAT_WAITING, STAT_FAIL, STAT_STARTED, STAT_FINISH_SUCCESS
+from .tasks.mail import TaskEmail
 
 log = logger.create()
 
+try:
+    from wand.image import Image
+    from wand.exceptions import MissingDelegateError
+    use_IM = True
+except (ImportError, RuntimeError) as e:
+    log.debug('Cannot import Image, generating covers from non jpg files will not work: %s', e)
+    use_IM = False
+    MissingDelegateError = BaseException
+
 
 # Convert existing book entry to new format
 def convert_book_format(book_id, calibrepath, old_book_format, new_book_format, user_id, kindle_mail=None):
     book = calibre_db.get_book(book_id)
     data = calibre_db.get_book_format(book.id, old_book_format)
+    file_path = os.path.join(calibrepath, book.path, data.name)
     if not data:
         error_message = _(u"%(format)s format not found for book id: %(book)d", format=old_book_format, book=book_id)
         log.error("convert_book_format: %s", error_message)
         return error_message
     if config.config_use_google_drive:
-        df = gd.getFileFromEbooksFolder(book.path, data.name + "." + old_book_format.lower())
-        if df:
-            datafile = os.path.join(calibrepath, book.path, data.name + u"." + old_book_format.lower())
-            if not os.path.exists(os.path.join(calibrepath, book.path)):
-                os.makedirs(os.path.join(calibrepath, book.path))
-            df.GetContentFile(datafile)
-        else:
+        if not gd.getFileFromEbooksFolder(book.path, data.name + "." + old_book_format.lower()):
             error_message = _(u"%(format)s not found on Google Drive: %(fn)s",
                               format=old_book_format, fn=data.name + "." + old_book_format.lower())
             return error_message
-    file_path = os.path.join(calibrepath, book.path, data.name)
-    if os.path.exists(file_path + "." + old_book_format.lower()):
-        # read settings and append converter task to queue
-        if kindle_mail:
-            settings = config.get_mail_settings()
-            settings['subject'] = _('Send to Kindle')  # pretranslate Subject for e-mail
-            settings['body'] = _(u'This e-mail has been sent via Calibre-Web.')
-            # text = _(u"%(format)s: %(book)s", format=new_book_format, book=book.title)
-        else:
-            settings = dict()
-        txt = (u"%s -> %s: %s" % (old_book_format, new_book_format, book.title))
-        settings['old_book_format'] = old_book_format
-        settings['new_book_format'] = new_book_format
-        worker.add_convert(file_path, book.id, user_id, txt, settings, kindle_mail)
-        return None
     else:
-        error_message = _(u"%(format)s not found: %(fn)s",
-                          format=old_book_format, fn=data.name + "." + old_book_format.lower())
-        return error_message
+        if not os.path.exists(file_path + "." + old_book_format.lower()):
+            error_message = _(u"%(format)s not found: %(fn)s",
+                              format=old_book_format, fn=data.name + "." + old_book_format.lower())
+            return error_message
+    # read settings and append converter task to queue
+    if kindle_mail:
+        settings = config.get_mail_settings()
+        settings['subject'] = _('Send to Kindle')  # pretranslate Subject for e-mail
+        settings['body'] = _(u'This e-mail has been sent via Calibre-Web.')
+    else:
+        settings = dict()
+    txt = (u"%s -> %s: %s" % (
+           old_book_format,
+           new_book_format,
+           "<a href=\"" + url_for('web.show_book', book_id=book.id) + "\">" + book.title + "</a>"))
+    settings['old_book_format'] = old_book_format
+    settings['new_book_format'] = new_book_format
+    WorkerThread.add(user_id, TaskConvert(file_path, book.id, txt, settings, kindle_mail, user_id))
+    return None
 
 
 def send_test_mail(kindle_mail, user_name):
-    worker.add_email(_(u'Calibre-Web test e-mail'), None, None,
-                     config.get_mail_settings(), kindle_mail, user_name,
-                     _(u"Test e-mail"), _(u'This e-mail has been sent via Calibre-Web.'))
+    WorkerThread.add(user_name, TaskEmail(_(u'Calibre-Web test e-mail'), None, None,
+                     config.get_mail_settings(), kindle_mail, _(u"Test e-mail"),
+                                          _(u'This e-mail has been sent via Calibre-Web.')))
     return
 
 
 # Send registration email or password reset email, depending on parameter resend (False means welcome email)
 def send_registration_mail(e_mail, user_name, default_password, resend=False):
-    text = "Hello %s!\r\n" % user_name
+    txt = "Hello %s!\r\n" % user_name
     if not resend:
-        text += "Your new account at Calibre-Web has been created. Thanks for joining us!\r\n"
-    text += "Please log in to your account using the following informations:\r\n"
-    text += "User name: %s\r\n" % user_name
-    text += "Password: %s\r\n" % default_password
-    text += "Don't forget to change your password after first login.\r\n"
-    text += "Sincerely\r\n\r\n"
-    text += "Your Calibre-Web team"
-    worker.add_email(_(u'Get Started with Calibre-Web'), None, None,
-                     config.get_mail_settings(), e_mail, None,
-                     _(u"Registration e-mail for user: %(name)s", name=user_name), text)
+        txt += "Your new account at Calibre-Web has been created. Thanks for joining us!\r\n"
+    txt += "Please log in to your account using the following informations:\r\n"
+    txt += "User name: %s\r\n" % user_name
+    txt += "Password: %s\r\n" % default_password
+    txt += "Don't forget to change your password after first login.\r\n"
+    txt += "Sincerely\r\n\r\n"
+    txt += "Your Calibre-Web team"
+    WorkerThread.add(None, TaskEmail(
+        subject=_(u'Get Started with Calibre-Web'),
+        filepath=None,
+        attachment=None,
+        settings=config.get_mail_settings(),
+        recipient=e_mail,
+        taskMessage=_(u"Registration e-mail for user: %(name)s", name=user_name),
+        text=txt
+    ))
     return
 
 
+def check_send_to_kindle_without_converter(entry):
+    bookformats = list()
+    # no converter - only for mobi and pdf formats
+    for ele in iter(entry.data):
+        if ele.uncompressed_size < config.mail_size:
+            if 'MOBI' in ele.format:
+                bookformats.append({'format': 'Mobi',
+                                    'convert': 0,
+                                    'text': _('Send %(format)s to Kindle', format='Mobi')})
+            if 'PDF' in ele.format:
+                bookformats.append({'format': 'Pdf',
+                                    'convert': 0,
+                                    'text': _('Send %(format)s to Kindle', format='Pdf')})
+            if 'AZW' in ele.format:
+                bookformats.append({'format': 'Azw',
+                                    'convert': 0,
+                                    'text': _('Send %(format)s to Kindle', format='Azw')})
+    return bookformats
+
+def check_send_to_kindle_with_converter(entry):
+    bookformats = list()
+    formats = list()
+    for ele in iter(entry.data):
+        if ele.uncompressed_size < config.mail_size:
+            formats.append(ele.format)
+    if 'MOBI' in formats:
+        bookformats.append({'format': 'Mobi',
+                            'convert': 0,
+                            'text': _('Send %(format)s to Kindle', format='Mobi')})
+    if 'AZW' in formats:
+        bookformats.append({'format': 'Azw',
+                            'convert': 0,
+                            'text': _('Send %(format)s to Kindle', format='Azw')})
+    if 'PDF' in formats:
+        bookformats.append({'format': 'Pdf',
+                            'convert': 0,
+                            'text': _('Send %(format)s to Kindle', format='Pdf')})
+    if 'EPUB' in formats and 'MOBI' not in formats:
+        bookformats.append({'format': 'Mobi',
+                            'convert': 1,
+                            'text': _('Convert %(orig)s to %(format)s and send to Kindle',
+                                      orig='Epub',
+                                      format='Mobi')})
+    if 'AZW3' in formats and not 'MOBI' in formats:
+        bookformats.append({'format': 'Mobi',
+                            'convert': 2,
+                            'text': _('Convert %(orig)s to %(format)s and send to Kindle',
+                                      orig='Azw3',
+                                      format='Mobi')})
+    return bookformats
+
+
 def check_send_to_kindle(entry):
     """
         returns all available book formats for sending to Kindle
     """
     if len(entry.data):
-        bookformats = list()
         if not config.config_converterpath:
-            # no converter - only for mobi and pdf formats
-            for ele in iter(entry.data):
-                if ele.uncompressed_size < config.mail_size:
-                    if 'MOBI' in ele.format:
-                        bookformats.append({'format': 'Mobi',
-                                            'convert': 0,
-                                            'text': _('Send %(format)s to Kindle', format='Mobi')})
-                    if 'PDF' in ele.format:
-                        bookformats.append({'format': 'Pdf',
-                                            'convert': 0,
-                                            'text': _('Send %(format)s to Kindle', format='Pdf')})
-                    if 'AZW' in ele.format:
-                        bookformats.append({'format': 'Azw',
-                                            'convert': 0,
-                                            'text': _('Send %(format)s to Kindle', format='Azw')})
+            book_formats = check_send_to_kindle_with_converter(entry)
         else:
-            formats = list()
-            for ele in iter(entry.data):
-                if ele.uncompressed_size < config.mail_size:
-                    formats.append(ele.format)
-            if 'MOBI' in formats:
-                bookformats.append({'format': 'Mobi',
-                                    'convert': 0,
-                                    'text': _('Send %(format)s to Kindle', format='Mobi')})
-            if 'AZW' in formats:
-                bookformats.append({'format': 'Azw',
-                                    'convert': 0,
-                                    'text': _('Send %(format)s to Kindle', format='Azw')})
-            if 'PDF' in formats:
-                bookformats.append({'format': 'Pdf',
-                                    'convert': 0,
-                                    'text': _('Send %(format)s to Kindle', format='Pdf')})
-            if config.config_converterpath:
-                if 'EPUB' in formats and not 'MOBI' in formats:
-                    bookformats.append({'format': 'Mobi',
-                                        'convert':1,
-                                        'text': _('Convert %(orig)s to %(format)s and send to Kindle',
-                                                  orig='Epub',
-                                                  format='Mobi')})
-                if 'AZW3' in formats and not 'MOBI' in formats:
-                    bookformats.append({'format': 'Mobi',
-                                        'convert': 2,
-                                        'text': _('Convert %(orig)s to %(format)s and send to Kindle',
-                                                  orig='Azw3',
-                                                  format='Mobi')})
-        return bookformats
+            book_formats = check_send_to_kindle_with_converter(entry)
+        return book_formats
     else:
         log.error(u'Cannot find book entry %d', entry.id)
         return None
@@ -221,9 +234,9 @@ def send_mail(book_id, book_format, convert, kindle_mail, calibrepath, user_id):
     for entry in iter(book.data):
         if entry.format.upper() == book_format.upper():
             converted_file_name = entry.name + '.' + book_format.lower()
-            worker.add_email(_(u"Send to Kindle"), book.path, converted_file_name,
-                             config.get_mail_settings(), kindle_mail, user_id,
-                             _(u"E-mail: %(book)s", book=book.title), _(u'This e-mail has been sent via Calibre-Web.'))
+            WorkerThread.add(user_id, TaskEmail(_(u"Send to Kindle"), book.path, converted_file_name,
+                             config.get_mail_settings(), kindle_mail,
+                             _(u"E-mail: %(book)s", book=book.title), _(u'This e-mail has been sent via Calibre-Web.')))
             return
     return _(u"The requested file could not be read. Maybe wrong permissions?")
 
@@ -343,66 +356,74 @@ def delete_book_file(book, calibrepath, book_format=None):
                                path=book.path)
 
 
-def update_dir_structure_file(book_id, calibrepath, first_author):
+# Moves files in file storage during author/title rename, or from temp dir to file storage
+def update_dir_structure_file(book_id, calibrepath, first_author, orignal_filepath, db_filename):
+    # get book database entry from id, if original path overwrite source with original_filepath
     localbook = calibre_db.get_book(book_id)
-    path = os.path.join(calibrepath, localbook.path)
+    if orignal_filepath:
+        path = orignal_filepath
+    else:
+        path = os.path.join(calibrepath, localbook.path)
 
+    # Create (current) authordir and titledir from database
     authordir = localbook.path.split('/')[0]
+    titledir = localbook.path.split('/')[1]
+
+    # Create new_authordir from parameter or from database
+    # Create new titledir from database and add id
     if first_author:
         new_authordir = get_valid_filename(first_author)
     else:
         new_authordir = get_valid_filename(localbook.authors[0].name)
-
-    titledir = localbook.path.split('/')[1]
     new_titledir = get_valid_filename(localbook.title) + " (" + str(book_id) + ")"
 
-    if titledir != new_titledir:
-        new_title_path = os.path.join(os.path.dirname(path), new_titledir)
+    if titledir != new_titledir or authordir != new_authordir or orignal_filepath:
+        new_path = os.path.join(calibrepath, new_authordir, new_titledir)
+        new_name = get_valid_filename(localbook.title) + ' - ' + get_valid_filename(new_authordir)
         try:
-            if not os.path.exists(new_title_path):
-                os.renames(os.path.normcase(path), os.path.normcase(new_title_path))
+            if orignal_filepath:
+                if not os.path.isdir(new_path):
+                    os.makedirs(new_path)
+                shutil.move(os.path.normcase(path), os.path.normcase(os.path.join(new_path, db_filename)))
+                log.debug("Moving title: %s to %s/%s", path, new_path, new_name)
+                # Check new path is not valid path
             else:
-                log.info("Copying title: %s into existing: %s", path, new_title_path)
-                for dir_name, __, file_list in os.walk(path):
-                    for file in file_list:
-                        os.renames(os.path.normcase(os.path.join(dir_name, file)),
-                                   os.path.normcase(os.path.join(new_title_path + dir_name[len(path):], file)))
-            path = new_title_path
-            localbook.path = localbook.path.split('/')[0] + '/' + new_titledir
+                if not os.path.exists(new_path):
+                    # move original path to new path
+                    log.debug("Moving title: %s to %s", path, new_path)
+                    shutil.move(os.path.normcase(path), os.path.normcase(new_path))
+                else: # path is valid copy only files to new location (merge)
+                    log.info("Moving title: %s into existing: %s", path, new_path)
+                    # Take all files and subfolder from old path (strange command)
+                    for dir_name, __, file_list in os.walk(path):
+                        for file in file_list:
+                            shutil.move(os.path.normcase(os.path.join(dir_name, file)),
+                                            os.path.normcase(os.path.join(new_path + dir_name[len(path):], file)))
+                            # os.unlink(os.path.normcase(os.path.join(dir_name, file)))
+            # change location in database to new author/title path
+            localbook.path = os.path.join(new_authordir, new_titledir).replace('\\','/')
         except OSError as ex:
-            log.error("Rename title from: %s to %s: %s", path, new_title_path, ex)
+            log.error("Rename title from: %s to %s: %s", path, new_path, ex)
             log.debug(ex, exc_info=True)
             return _("Rename title from: '%(src)s' to '%(dest)s' failed with error: %(error)s",
-                     src=path, dest=new_title_path, error=str(ex))
-    if authordir != new_authordir:
-        new_author_path = os.path.join(calibrepath, new_authordir, os.path.basename(path))
+                     src=path, dest=new_path, error=str(ex))
+
+        # Rename all files from old names to new names
         try:
-            os.renames(os.path.normcase(path), os.path.normcase(new_author_path))
-            localbook.path = new_authordir + '/' + localbook.path.split('/')[1]
-        except OSError as ex:
-            log.error("Rename author from: %s to %s: %s", path, new_author_path, ex)
-            log.debug(ex, exc_info=True)
-            return _("Rename author from: '%(src)s' to '%(dest)s' failed with error: %(error)s",
-                     src=path, dest=new_author_path, error=str(ex))
-    # Rename all files from old names to new names
-    if authordir != new_authordir or titledir != new_titledir:
-        new_name = ""
-        try:
-            new_name = get_valid_filename(localbook.title) + ' - ' + get_valid_filename(new_authordir)
-            path_name = os.path.join(calibrepath, new_authordir, os.path.basename(path))
             for file_format in localbook.data:
-                os.renames(os.path.normcase(
-                    os.path.join(path_name, file_format.name + '.' + file_format.format.lower())),
-                           os.path.normcase(os.path.join(path_name, new_name + '.' + file_format.format.lower())))
+                shutil.move(os.path.normcase(
+                    os.path.join(new_path, file_format.name + '.' + file_format.format.lower())),
+                    os.path.normcase(os.path.join(new_path, new_name + '.' + file_format.format.lower())))
                 file_format.name = new_name
+            if not orignal_filepath and len(os.listdir(os.path.dirname(path))) == 0:
+                shutil.rmtree(os.path.dirname(path))
         except OSError as ex:
-            log.error("Rename file in path %s to %s: %s", path, new_name, ex)
+            log.error("Rename file in path %s to %s: %s", new_path, new_name, ex)
             log.debug(ex, exc_info=True)
             return _("Rename file in path '%(src)s' to '%(dest)s' failed with error: %(error)s",
-                     src=path, dest=new_name, error=str(ex))
+                     src=new_path, dest=new_name, error=str(ex))
     return False
 
-
 def update_dir_structure_gdrive(book_id, first_author):
     error = False
     book = calibre_db.get_book(book_id)
@@ -505,11 +526,11 @@ def uniq(inpt):
 # ################################# External interface #################################
 
 
-def update_dir_stucture(book_id, calibrepath, first_author=None):
+def update_dir_stucture(book_id, calibrepath, first_author=None, orignal_filepath=None, db_filename=None):
     if config.config_use_google_drive:
         return update_dir_structure_gdrive(book_id, first_author)
     else:
-        return update_dir_structure_file(book_id, calibrepath, first_author)
+        return update_dir_structure_file(book_id, calibrepath, first_author, orignal_filepath, db_filename)
 
 
 def delete_book(book, calibrepath, book_format):
@@ -550,8 +571,7 @@ def get_book_cover_internal(book, use_generic_cover_on_failure):
                     log.error('%s/cover.jpg not found on Google Drive', book.path)
                     return get_cover_on_failure(use_generic_cover_on_failure)
             except Exception as e:
-                log.exception(e)
-                # traceback.print_exc()
+                log.debug_or_exception(e)
                 return get_cover_on_failure(use_generic_cover_on_failure)
         else:
             cover_file_path = os.path.join(config.config_calibre_dir, book.path)
@@ -574,29 +594,35 @@ def save_cover_from_url(url, book_path):
             requests.exceptions.Timeout) as ex:
         log.info(u'Cover Download Error %s', ex)
         return False, _("Error Downloading Cover")
-    except UnidentifiedImageError as ex:
+    except MissingDelegateError as ex:
         log.info(u'File Format Error %s', ex)
         return False, _("Cover Format Error")
 
 
 def save_cover_from_filestorage(filepath, saved_filename, img):
-    if hasattr(img, '_content'):
-        f = open(os.path.join(filepath, saved_filename), "wb")
-        f.write(img._content)
-        f.close()
-    else:
-        # check if file path exists, otherwise create it, copy file to calibre path and delete temp file
-        if not os.path.exists(filepath):
-            try:
-                os.makedirs(filepath)
-            except OSError:
-                log.error(u"Failed to create path for cover")
-                return False, _(u"Failed to create path for cover")
+    # check if file path exists, otherwise create it, copy file to calibre path and delete temp file
+    if not os.path.exists(filepath):
         try:
-            img.save(os.path.join(filepath, saved_filename))
-        except (IOError, OSError):
-            log.error(u"Cover-file is not a valid image file, or could not be stored")
-            return False, _(u"Cover-file is not a valid image file, or could not be stored")
+            os.makedirs(filepath)
+        except OSError:
+            log.error(u"Failed to create path for cover")
+            return False, _(u"Failed to create path for cover")
+    try:
+        # upload of jgp file without wand
+        if isinstance(img, requests.Response):
+            with open(os.path.join(filepath, saved_filename), 'wb') as f:
+                f.write(img.content)
+        else:
+            if hasattr(img, "metadata"):
+                # upload of jpg/png... via url
+                img.save(filename=os.path.join(filepath, saved_filename))
+                img.close()
+            else:
+                # upload of jpg/png... from hdd
+                img.save(os.path.join(filepath, saved_filename))
+    except (IOError, OSError):
+        log.error(u"Cover-file is not a valid image file, or could not be stored")
+        return False, _(u"Cover-file is not a valid image file, or could not be stored")
     return True, None
 
 
@@ -604,31 +630,33 @@ def save_cover_from_filestorage(filepath, saved_filename, img):
 def save_cover(img, book_path):
     content_type = img.headers.get('content-type')
 
-    if use_PIL:
-        if content_type not in ('image/jpeg', 'image/png', 'image/webp'):
-            log.error("Only jpg/jpeg/png/webp files are supported as coverfile")
-            return False, _("Only jpg/jpeg/png/webp files are supported as coverfile")
+    if use_IM:
+        if content_type not in ('image/jpeg', 'image/png', 'image/webp', 'image/bmp'):
+            log.error("Only jpg/jpeg/png/webp/bmp files are supported as coverfile")
+            return False, _("Only jpg/jpeg/png/webp/bmp files are supported as coverfile")
         # convert to jpg because calibre only supports jpg
-        if content_type in ('image/png', 'image/webp'):
+        if content_type != 'image/jpg':
             if hasattr(img, 'stream'):
-                imgc = PILImage.open(img.stream)
+                imgc = Image(blob=img.stream)
             else:
-                imgc = PILImage.open(io.BytesIO(img.content))
-            im = imgc.convert('RGB')
-            tmp_bytesio = io.BytesIO()
-            im.save(tmp_bytesio, format='JPEG')
-            img._content = tmp_bytesio.getvalue()
+                imgc = Image(blob=io.BytesIO(img.content))
+            imgc.format = 'jpeg'
+            imgc.transform_colorspace("rgb")
+            img = imgc
     else:
         if content_type not in 'image/jpeg':
             log.error("Only jpg/jpeg files are supported as coverfile")
             return False, _("Only jpg/jpeg files are supported as coverfile")
 
     if config.config_use_google_drive:
-        tmpDir = gettempdir()
-        ret, message = save_cover_from_filestorage(tmpDir, "uploaded_cover.jpg", img)
+        tmp_dir = os.path.join(gettempdir(), 'calibre_web')
+
+        if not os.path.isdir(tmp_dir):
+            os.mkdir(tmp_dir)
+        ret, message = save_cover_from_filestorage(tmp_dir, "uploaded_cover.jpg", img)
         if ret is True:
-            gd.uploadFileToEbooksFolder(os.path.join(book_path, 'cover.jpg'),
-                                        os.path.join(tmpDir, "uploaded_cover.jpg"))
+            gd.uploadFileToEbooksFolder(os.path.join(book_path, 'cover.jpg').replace("\\","/"),
+                                        os.path.join(tmp_dir, "uploaded_cover.jpg"))
             log.info("Cover is saved on Google Drive")
             return True, None
         else:
@@ -682,7 +710,7 @@ def check_unrar(unrarLocation):
                 log.debug("unrar version %s", version)
                 break
     except (OSError, UnicodeDecodeError) as err:
-        log.exception(err)
+        log.debug_or_exception(err)
         return _('Error excecuting UnRar')
 
 
@@ -722,47 +750,30 @@ def format_runtime(runtime):
 # helper function to apply localize status information in tasklist entries
 def render_task_status(tasklist):
     renderedtasklist = list()
-    for task in tasklist:
-        if task['user'] == current_user.nickname or current_user.role_admin():
-            if task['formStarttime']:
-                task['starttime'] = format_datetime(task['formStarttime'], format='short', locale=get_locale())
-            # task2['formStarttime'] = ""
-            else:
-                if 'starttime' not in task:
-                    task['starttime'] = ""
-
-            if 'formRuntime' not in task:
-                task['runtime'] = ""
-            else:
-                task['runtime'] = format_runtime(task['formRuntime'])
+    for __, user, added, task in tasklist:
+        if user == current_user.nickname or current_user.role_admin():
+            ret = {}
+            if task.start_time:
+                ret['starttime'] = format_datetime(task.start_time, format='short', locale=get_locale())
+                ret['runtime'] = format_runtime(task.runtime)
 
             # localize the task status
-            if isinstance(task['stat'], int):
-                if task['stat'] == STAT_WAITING:
-                    task['status'] = _(u'Waiting')
-                elif task['stat'] == STAT_FAIL:
-                    task['status'] = _(u'Failed')
-                elif task['stat'] == STAT_STARTED:
-                    task['status'] = _(u'Started')
-                elif task['stat'] == STAT_FINISH_SUCCESS:
-                    task['status'] = _(u'Finished')
+            if isinstance(task.stat, int):
+                if task.stat == STAT_WAITING:
+                    ret['status'] = _(u'Waiting')
+                elif task.stat == STAT_FAIL:
+                    ret['status'] = _(u'Failed')
+                elif task.stat == STAT_STARTED:
+                    ret['status'] = _(u'Started')
+                elif task.stat == STAT_FINISH_SUCCESS:
+                    ret['status'] = _(u'Finished')
                 else:
-                    task['status'] = _(u'Unknown Status')
+                    ret['status'] = _(u'Unknown Status')
 
-            # localize the task type
-            if isinstance(task['taskType'], int):
-                if task['taskType'] == TASK_EMAIL:
-                    task['taskMessage'] = _(u'E-mail: ') + task['taskMess']
-                elif task['taskType'] == TASK_CONVERT:
-                    task['taskMessage'] = _(u'Convert: ') + task['taskMess']
-                elif task['taskType'] == TASK_UPLOAD:
-                    task['taskMessage'] = _(u'Upload: ') + task['taskMess']
-                elif task['taskType'] == TASK_CONVERT_ANY:
-                    task['taskMessage'] = _(u'Convert: ') + task['taskMess']
-                else:
-                    task['taskMessage'] = _(u'Unknown Task: ') + task['taskMess']
-
-            renderedtasklist.append(task)
+            ret['taskMessage'] = "{}: {}".format(_(task.name), task.message)
+            ret['progress'] = "{} %".format(int(task.progress * 100))
+            ret['user'] = user
+            renderedtasklist.append(ret)
 
     return renderedtasklist
 

cps/isoLanguages.py

@@ -66,3 +66,27 @@ def get_language_codes(locale, language_names, remainder=None):
     if remainder is not None:
         remainder.extend(language_names)
     return languages
+
+def get_valid_language_codes(locale, language_names, remainder=None):
+    languages = list()
+    if "" in language_names:
+        language_names.remove("")
+    for k, __ in get_language_names(locale).items():
+        if k in language_names:
+            languages.append(k)
+            language_names.remove(k)
+    if remainder is not None and len(language_names):
+        remainder.extend(language_names)
+    return languages
+
+def get_lang3(lang):
+    try:
+        if len(lang) == 2:
+            ret_value = get(part1=lang).part3
+        elif len(lang) == 3:
+            ret_value = lang
+        else:
+            ret_value = ""
+    except KeyError:
+        ret_value = lang
+    return ret_value

cps/jinjia.py

@@ -25,7 +25,7 @@
 from __future__ import division, print_function, unicode_literals
 import datetime
 import mimetypes
-import re
+from uuid import uuid4
 
 from babel.dates import format_date
 from flask import Blueprint, request, url_for
@@ -44,6 +44,8 @@ log = logger.create()
 def url_for_other_page(page):
     args = request.view_args.copy()
     args['page'] = page
+    for get, val in request.args.items():
+        args[get] = val
     return url_for(request.endpoint, **args)
 
 
@@ -76,22 +78,18 @@ def mimetype_filter(val):
 @jinjia.app_template_filter('formatdate')
 def formatdate_filter(val):
     try:
-        conformed_timestamp = re.sub(r"[:]|([-](?!((\d{2}[:]\d{2})|(\d{4}))$))", '', val)
-        formatdate = datetime.datetime.strptime(conformed_timestamp[:15], "%Y%m%d %H%M%S")
-        return format_date(formatdate, format='medium', locale=get_locale())
+        return format_date(val, format='medium', locale=get_locale())
     except AttributeError as e:
         log.error('Babel error: %s, Current user locale: %s, Current User: %s', e,
                   current_user.locale,
                   current_user.nickname
                   )
-        return formatdate
+        return val
 
 
 @jinjia.app_template_filter('formatdateinput')
 def format_date_input(val):
-    conformed_timestamp = re.sub(r"[:]|([-](?!((\d{2}[:]\d{2})|(\d{4}))$))", '', val)
-    date_obj = datetime.datetime.strptime(conformed_timestamp[:15], "%Y%m%d %H%M%S")
-    input_date = date_obj.isoformat().split('T', 1)[0]  # Hack to support dates <1900
+    input_date = val.isoformat().split('T', 1)[0]  # Hack to support dates <1900
     return '' if input_date == "0101-01-01" else input_date
 
 
@@ -112,9 +110,26 @@ def timestamptodate(date, fmt=None):
 def yesno(value, yes, no):
     return yes if value else no
 
+
 @jinjia.app_template_filter('formatfloat')
 def formatfloat(value, decimals=1):
     formatedstring = '%d' % value
     if (value % 1) != 0:
         formatedstring = ('%s.%d' % (formatedstring, (value % 1) * 10**decimals)).rstrip('0')
     return formatedstring
+
+
+@jinjia.app_template_filter('formatseriesindex')
+def formatseriesindex_filter(series_index):
+    if series_index:
+        if int(series_index) - series_index == 0:
+            return int(series_index)
+        else:
+            return series_index
+    return 0
+
+@jinjia.app_template_filter('uuidfilter')
+def uuidfilter(var):
+    return uuid4()
+
+

cps/kobo.py

@@ -39,22 +39,25 @@ from flask import (
     redirect,
     abort
 )
-from flask_login import current_user, login_required
+from flask_login import current_user
 from werkzeug.datastructures import Headers
 from sqlalchemy import func
-from sqlalchemy.sql.expression import and_, or_
+from sqlalchemy.sql.expression import and_
 from sqlalchemy.exc import StatementError
 import requests
 
 from . import config, logger, kobo_auth, db, calibre_db, helper, shelf as shelf_lib, ub
+from .helper import get_download_link
 from .services import SyncToken as SyncToken
 from .web import download_required
-from .kobo_auth import requires_kobo_auth
+from .kobo_auth import requires_kobo_auth, get_auth_token
 
 KOBO_FORMATS = {"KEPUB": ["KEPUB"], "EPUB": ["EPUB3", "EPUB"]}
 KOBO_STOREAPI_URL = "https://storeapi.kobo.com"
 KOBO_IMAGEHOST_URL = "https://kbimages1-a.akamaihd.net"
 
+SYNC_ITEM_LIMIT = 100
+
 kobo = Blueprint("kobo", __name__, url_prefix="/kobo/<auth_token>")
 kobo_auth.disable_failed_auth_redirect_for_blueprint(kobo)
 kobo_auth.register_url_value_preprocessor(kobo)
@@ -119,7 +122,11 @@ def redirect_or_proxy_request():
 
 
 def convert_to_kobo_timestamp_string(timestamp):
-    return timestamp.strftime("%Y-%m-%dT%H:%M:%SZ")
+    try:
+        return timestamp.strftime("%Y-%m-%dT%H:%M:%SZ")
+    except AttributeError as exc:
+        log.debug("Timestamp not valid: {}".format(exc))
+        return datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
 
 
 @kobo.route("/v1/library/sync")
@@ -137,68 +144,84 @@ def HandleSyncRequest():
     new_books_last_modified = sync_token.books_last_modified
     new_books_last_created = sync_token.books_last_created
     new_reading_state_last_modified = sync_token.reading_state_last_modified
+    new_archived_last_modified = datetime.datetime.min
     sync_results = []
 
     # We reload the book database so that the user get's a fresh view of the library
     # in case of external changes (e.g: adding a book through Calibre).
     calibre_db.reconnect_db(config, ub.app_DB_path)
 
-    archived_books = (
-        ub.session.query(ub.ArchivedBook)
-        .filter(ub.ArchivedBook.user_id == int(current_user.id))
-        .all()
-    )
-
-    # We join-in books that have had their Archived bit recently modified in order to either:
-    #   * Restore them to the user's device.
-    #   * Delete them from the user's device.
-    # (Ideally we would use a join for this logic, however cross-database joins don't look trivial in SqlAlchemy.)
-    recently_restored_or_archived_books = []
-    archived_book_ids = {}
-    new_archived_last_modified = datetime.datetime.min
-    for archived_book in archived_books:
-        if archived_book.last_modified > sync_token.archive_last_modified:
-            recently_restored_or_archived_books.append(archived_book.book_id)
-        if archived_book.is_archived:
-            archived_book_ids[archived_book.book_id] = True
-        new_archived_last_modified = max(
-            new_archived_last_modified, archived_book.last_modified)
-
-    # sqlite gives unexpected results when performing the last_modified comparison without the datetime cast.
-    # It looks like it's treating the db.Books.last_modified field as a string and may fail
-    # the comparison because of the +00:00 suffix.
-    changed_entries = (
-        calibre_db.session.query(db.Books)
-        .join(db.Data)
-        .filter(or_(func.datetime(db.Books.last_modified) > sync_token.books_last_modified,
-                    db.Books.id.in_(recently_restored_or_archived_books)))
-        .filter(db.Data.format.in_(KOBO_FORMATS))
-        .all()
-    )
+    if sync_token.books_last_id > -1:
+        changed_entries = (
+            calibre_db.session.query(db.Books, ub.ArchivedBook.last_modified, ub.ArchivedBook.is_archived)
+            .join(db.Data).outerjoin(ub.ArchivedBook, db.Books.id == ub.ArchivedBook.book_id)
+            .filter(db.Books.last_modified >= sync_token.books_last_modified)
+            .filter(db.Books.id>sync_token.books_last_id)
+            .filter(db.Data.format.in_(KOBO_FORMATS))
+            .order_by(db.Books.last_modified)
+            .order_by(db.Books.id)
+            .limit(SYNC_ITEM_LIMIT)
+        )
+    else:
+        changed_entries = (
+            calibre_db.session.query(db.Books, ub.ArchivedBook.last_modified, ub.ArchivedBook.is_archived)
+            .join(db.Data).outerjoin(ub.ArchivedBook, db.Books.id == ub.ArchivedBook.book_id)
+            .filter(db.Books.last_modified > sync_token.books_last_modified)
+            .filter(db.Data.format.in_(KOBO_FORMATS))
+            .order_by(db.Books.last_modified)
+            .order_by(db.Books.id)
+            .limit(SYNC_ITEM_LIMIT)
+        )
 
     reading_states_in_new_entitlements = []
     for book in changed_entries:
-        kobo_reading_state = get_or_create_reading_state(book.id)
+        formats = [data.format for data in book.Books.data]
+        if not 'KEPUB' in formats and config.config_kepubifypath and 'EPUB' in formats:
+            helper.convert_book_format(book.Books.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.nickname)
+
+        kobo_reading_state = get_or_create_reading_state(book.Books.id)
         entitlement = {
-            "BookEntitlement": create_book_entitlement(book, archived=(book.id in archived_book_ids)),
-            "BookMetadata": get_metadata(book),
+            "BookEntitlement": create_book_entitlement(book.Books, archived=(book.is_archived == True)),
+            "BookMetadata": get_metadata(book.Books),
         }
 
         if kobo_reading_state.last_modified > sync_token.reading_state_last_modified:
-            entitlement["ReadingState"] = get_kobo_reading_state_response(book, kobo_reading_state)
+            entitlement["ReadingState"] = get_kobo_reading_state_response(book.Books, kobo_reading_state)
             new_reading_state_last_modified = max(new_reading_state_last_modified, kobo_reading_state.last_modified)
-            reading_states_in_new_entitlements.append(book.id)
+            reading_states_in_new_entitlements.append(book.Books.id)
 
-        if book.timestamp > sync_token.books_last_created:
+        if book.Books.timestamp > sync_token.books_last_created:
             sync_results.append({"NewEntitlement": entitlement})
         else:
             sync_results.append({"ChangedEntitlement": entitlement})
 
         new_books_last_modified = max(
-            book.last_modified, new_books_last_modified
+            book.Books.last_modified, new_books_last_modified
         )
-        new_books_last_created = max(book.timestamp, new_books_last_created)
+        new_books_last_created = max(book.Books.timestamp, new_books_last_created)
 
+    max_change = (changed_entries
+        .from_self()
+        .filter(ub.ArchivedBook.is_archived)
+        .order_by(func.datetime(ub.ArchivedBook.last_modified).desc())
+        .first()
+    )
+    if max_change:
+        max_change = max_change.last_modified
+    else:
+        max_change = new_archived_last_modified
+    new_archived_last_modified = max(new_archived_last_modified, max_change)
+
+    # no. of books returned
+    book_count = changed_entries.count()
+
+    # last entry:
+    if book_count:
+        books_last_id = changed_entries.all()[-1].Books.id or -1
+    else:
+        books_last_id = -1
+
+    # generate reading state data
     changed_reading_states = (
         ub.session.query(ub.KoboReadingState)
         .filter(and_(func.datetime(ub.KoboReadingState.last_modified) > sync_token.reading_state_last_modified,
@@ -220,11 +243,12 @@ def HandleSyncRequest():
     sync_token.books_last_modified = new_books_last_modified
     sync_token.archive_last_modified = new_archived_last_modified
     sync_token.reading_state_last_modified = new_reading_state_last_modified
+    sync_token.books_last_id = books_last_id
 
-    return generate_sync_response(sync_token, sync_results)
+    return generate_sync_response(sync_token, sync_results, book_count)
 
 
-def generate_sync_response(sync_token, sync_results):
+def generate_sync_response(sync_token, sync_results, set_cont=False):
     extra_headers = {}
     if config.config_kobo_proxy:
         # Merge in sync results from the official Kobo store.
@@ -240,6 +264,8 @@ def generate_sync_response(sync_token, sync_results):
 
         except Exception as e:
             log.error("Failed to receive or parse response from Kobo's sync endpoint: " + str(e))
+    if set_cont:
+        extra_headers["x-kobo-sync"] = "continue"
     sync_token.to_headers(extra_headers)
 
     response = make_response(jsonify(sync_results), extra_headers)
@@ -270,10 +296,11 @@ def get_download_url_for_book(book, book_format):
         else:
             host = request.host
 
-        return "{url_scheme}://{url_base}:{url_port}/download/{book_id}/{book_format}".format(
+        return "{url_scheme}://{url_base}:{url_port}/kobo/{auth_token}/download/{book_id}/{book_format}".format(
             url_scheme=request.scheme,
             url_base=host,
             url_port=config.config_external_port,
+            auth_token=get_auth_token(),
             book_id=book.id,
             book_format=book_format.lower()
         )
@@ -344,7 +371,9 @@ def get_seriesindex(book):
 
 def get_metadata(book):
     download_urls = []
-    for book_data in book.data:
+    kepub = [data for data in book.data if data.format == 'KEPUB']
+
+    for book_data in kepub if len(kepub) > 0 else book.data:
         if book_data.format not in KOBO_FORMATS:
             continue
         for kobo_format in KOBO_FORMATS[book_data.format]:
@@ -435,8 +464,7 @@ def HandleTagCreate():
     items_unknown_to_calibre = add_items_to_shelf(items, shelf)
     if items_unknown_to_calibre:
         log.debug("Received request to add unknown books to a collection. Silently ignoring items.")
-    ub.session.commit()
-
+    ub.session_commit()
     return make_response(jsonify(str(shelf.uuid)), 201)
 
 
@@ -468,7 +496,7 @@ def HandleTagUpdate(tag_id):
 
         shelf.name = name
         ub.session.merge(shelf)
-        ub.session.commit()
+        ub.session_commit()
     return make_response(' ', 200)
 
 
@@ -520,8 +548,7 @@ def HandleTagAddItem(tag_id):
         log.debug("Received request to add an unknown book to a collection. Silently ignoring item.")
 
     ub.session.merge(shelf)
-    ub.session.commit()
-
+    ub.session_commit()
     return make_response('', 201)
 
 
@@ -561,7 +588,7 @@ def HandleTagRemoveItem(tag_id):
             shelf.books.filter(ub.BookShelf.book_id == book.id).delete()
         except KeyError:
             items_unknown_to_calibre.append(item)
-    ub.session.commit()
+    ub.session_commit()
 
     if items_unknown_to_calibre:
         log.debug("Received request to remove an unknown book to a collecition. Silently ignoring item.")
@@ -607,7 +634,7 @@ def sync_shelves(sync_token, sync_results):
                 "ChangedTag": tag
             })
     sync_token.tags_last_modified = new_tags_last_modified
-    ub.session.commit()
+    ub.session_commit()
 
 
 # Creates a Kobo "Tag" object from a ub.Shelf object
@@ -688,7 +715,7 @@ def HandleStateRequest(book_uuid):
             abort(400, description="Malformed request data is missing 'ReadingStates' key")
 
         ub.session.merge(kobo_reading_state)
-        ub.session.commit()
+        ub.session_commit()
         return jsonify({
             "RequestResult": "Success",
             "UpdateResults": [update_results_response],
@@ -726,7 +753,7 @@ def get_or_create_reading_state(book_id):
         kobo_reading_state.statistics = ub.KoboStatistics()
         book_read.kobo_reading_state = kobo_reading_state
     ub.session.add(book_read)
-    ub.session.commit()
+    ub.session_commit()
     return book_read.kobo_reading_state
 
 
@@ -829,8 +856,7 @@ def HandleBookDeletionRequest(book_uuid):
     archived_book.last_modified = datetime.datetime.utcnow()
 
     ub.session.merge(archived_book)
-    ub.session.commit()
-
+    ub.session_commit()
     return ("", 204)
 
 
@@ -856,6 +882,7 @@ def HandleUserRequest(dummy=None):
 @kobo.route("/v1/products/<dummy>/recommendations", methods=["GET", "POST"])
 @kobo.route("/v1/products/<dummy>/nextread", methods=["GET", "POST"])
 @kobo.route("/v1/products/<dummy>/reviews", methods=["GET", "POST"])
+@kobo.route("/v1/products/books/external/<dummy>", methods=["GET", "POST"])
 @kobo.route("/v1/products/books/series/<dummy>", methods=["GET", "POST"])
 @kobo.route("/v1/products/books/<dummy>", methods=["GET", "POST"])
 @kobo.route("/v1/products/dailydeal", methods=["GET", "POST"])
@@ -865,17 +892,6 @@ def HandleProductsRequest(dummy=None):
     return redirect_or_proxy_request()
 
 
-'''@kobo.errorhandler(404)
-def handle_404(err):
-    # This handler acts as a catch-all for endpoints that we don't have an interest in
-    # implementing (e.g: v1/analytics/gettests, v1/user/recommendations, etc)
-    if err:
-        print('404')
-        return jsonify(error=str(err)), 404
-    log.debug("Unknown Request received: %s, method: %s, data: %s", request.base_url, request.method, request.data)
-    return redirect_or_proxy_request()'''
-
-
 def make_calibre_web_auth_response():
     # As described in kobo_auth.py, CalibreWeb doesn't make use practical use of this auth/device API call for
     # authentation (nor for authorization). We return a dummy response just to keep the device happy.
@@ -919,7 +935,7 @@ def HandleInitRequest():
             store_response_json = store_response.json()
             if "Resources" in store_response_json:
                 kobo_resources = store_response_json["Resources"]
-        except:
+        except Exception:
             log.error("Failed to receive or parse response from Kobo's init endpoint. Falling back to un-proxied mode.")
     if not kobo_resources:
         kobo_resources = NATIVE_KOBO_RESOURCES()
@@ -976,6 +992,13 @@ def HandleInitRequest():
     return response
 
 
+@kobo.route("/download/<book_id>/<book_format>")
+@requires_kobo_auth
+@download_required
+def download_book(book_id, book_format):
+    return get_download_link(book_id, book_format, "kobo")
+
+
 def NATIVE_KOBO_RESOURCES():
     return {
         "account_page": "https://secure.kobobooks.com/profile",

cps/kobo_auth.py

@@ -64,11 +64,11 @@ from datetime import datetime
 from os import urandom
 
 from flask import g, Blueprint, url_for, abort, request
-from flask_login import login_user, login_required
+from flask_login import login_user, current_user, login_required
 from flask_babel import gettext as _
 
-from . import logger, ub, lm
-from .web import render_title_template
+from . import logger, config, calibre_db, db, helper, ub, lm
+from .render_template import render_title_template
 
 try:
     from functools import wraps
@@ -147,7 +147,15 @@ def generate_auth_token(user_id):
             auth_token.token_type = 1
 
             ub.session.add(auth_token)
-            ub.session.commit()
+            ub.session_commit()
+
+        books = calibre_db.session.query(db.Books).join(db.Data).all()
+
+        for book in books:
+            formats = [data.format for data in book.data]
+            if not 'KEPUB' in formats and config.config_kepubifypath and 'EPUB' in formats:
+                helper.convert_book_format(book.id, config.config_calibre_dir, 'EPUB', 'KEPUB', current_user.nickname)
+
         return render_title_template(
             "generate_kobo_auth_url.html",
             title=_(u"Kobo Setup"),
@@ -164,5 +172,5 @@ def delete_auth_token(user_id):
     # Invalidate any prevously generated Kobo Auth token for this user.
     ub.session.query(ub.RemoteAuthToken).filter(ub.RemoteAuthToken.user_id == user_id)\
         .filter(ub.RemoteAuthToken.token_type==1).delete()
-    ub.session.commit()
-    return ""
+
+    return ub.session_commit()

cps/logger.py

@@ -41,10 +41,37 @@ logging.addLevelName(logging.WARNING, "WARN")
 logging.addLevelName(logging.CRITICAL, "CRIT")
 
 
+class _Logger(logging.Logger):
+
+    def debug_or_exception(self, message, *args, **kwargs):
+        if sys.version_info > (3, 7):
+            if is_debug_enabled():
+                self.exception(message, stacklevel=2, *args, **kwargs)
+            else:
+                self.error(message, stacklevel=2, *args, **kwargs)
+        elif sys.version_info > (3, 0):
+            if is_debug_enabled():
+                self.exception(message, stack_info=True, *args, **kwargs)
+            else:
+                self.error(message, *args, **kwargs)
+        else:
+            if is_debug_enabled():
+                self.exception(message, *args, **kwargs)
+            else:
+                self.error(message, *args, **kwargs)
+
+
+    def debug_no_auth(self, message, *args, **kwargs):
+        if message.startswith("send: AUTH"):
+            self.debug(message[:16], stacklevel=2, *args, **kwargs)
+        else:
+            self.debug(message, stacklevel=2, *args, **kwargs)
+
+
+
 def get(name=None):
     return logging.getLogger(name)
 
-
 def create():
     parent_frame = inspect.stack(0)[1]
     if hasattr(parent_frame, 'frame'):
@@ -54,7 +81,6 @@ def create():
     parent_module = inspect.getmodule(parent_frame)
     return get(parent_module.__name__)
 
-
 def is_debug_enabled():
     return logging.root.level <= logging.DEBUG
 
@@ -99,6 +125,7 @@ def setup(log_file, log_level=None):
     May be called multiple times.
     '''
     log_level = log_level or DEFAULT_LOG_LEVEL
+    logging.setLoggerClass(_Logger)
     logging.getLogger(__package__).setLevel(log_level)
 
     r = logging.root
@@ -126,11 +153,11 @@ def setup(log_file, log_level=None):
             file_handler.baseFilename = log_file
     else:
         try:
-            file_handler = RotatingFileHandler(log_file, maxBytes=50000, backupCount=2)
+            file_handler = RotatingFileHandler(log_file, maxBytes=50000, backupCount=2, encoding='utf-8')
         except IOError:
             if log_file == DEFAULT_LOG_FILE:
                 raise
-            file_handler = RotatingFileHandler(DEFAULT_LOG_FILE, maxBytes=50000, backupCount=2)
+            file_handler = RotatingFileHandler(DEFAULT_LOG_FILE, maxBytes=50000, backupCount=2, encoding='utf-8')
             log_file = ""
     file_handler.setFormatter(FORMATTER)
 
@@ -152,11 +179,11 @@ def create_access_log(log_file, log_name, formatter):
     access_log.propagate = False
     access_log.setLevel(logging.INFO)
     try:
-        file_handler = RotatingFileHandler(log_file, maxBytes=50000, backupCount=2)
+        file_handler = RotatingFileHandler(log_file, maxBytes=50000, backupCount=2, encoding='utf-8')
     except IOError:
         if log_file == DEFAULT_ACCESS_LOG:
             raise
-        file_handler = RotatingFileHandler(DEFAULT_ACCESS_LOG, maxBytes=50000, backupCount=2)
+        file_handler = RotatingFileHandler(DEFAULT_ACCESS_LOG, maxBytes=50000, backupCount=2, encoding='utf-8')
         log_file = ""
 
     file_handler.setFormatter(formatter)

cps/oauth.py

@@ -19,7 +19,6 @@
 from __future__ import division, print_function, unicode_literals
 from flask import session
 
-
 try:
     from flask_dance.consumer.backend.sqla import SQLAlchemyBackend, first, _get_real_user
     from sqlalchemy.orm.exc import NoResultFound
@@ -34,134 +33,131 @@ except ImportError:
     except ImportError:
         pass
 
-try:
-    class OAuthBackend(SQLAlchemyBackend):
-        """
-        Stores and retrieves OAuth tokens using a relational database through
-        the `SQLAlchemy`_ ORM.
 
-        .. _SQLAlchemy: https://www.sqlalchemy.org/
-        """
-        def __init__(self, model, session, provider_id,
-                     user=None, user_id=None, user_required=None, anon_user=None,
-                     cache=None):
-            self.provider_id = provider_id
-            super(OAuthBackend, self).__init__(model, session, user, user_id, user_required, anon_user, cache)
+class OAuthBackend(SQLAlchemyBackend):
+    """
+    Stores and retrieves OAuth tokens using a relational database through
+    the `SQLAlchemy`_ ORM.
 
-        def get(self, blueprint, user=None, user_id=None):
-            if self.provider_id + '_oauth_token' in session and session[self.provider_id + '_oauth_token'] != '':
-                return session[self.provider_id + '_oauth_token']
-            # check cache
-            cache_key = self.make_cache_key(blueprint=blueprint, user=user, user_id=user_id)
-            token = self.cache.get(cache_key)
-            if token:
-                return token
-
-            # if not cached, make database queries
-            query = (
-                self.session.query(self.model)
-                .filter_by(provider=self.provider_id)
-            )
-            uid = first([user_id, self.user_id, blueprint.config.get("user_id")])
-            u = first(_get_real_user(ref, self.anon_user)
-                      for ref in (user, self.user, blueprint.config.get("user")))
-
-            use_provider_user_id = False
-            if self.provider_id + '_oauth_user_id' in session and session[self.provider_id + '_oauth_user_id'] != '':
-                query = query.filter_by(provider_user_id=session[self.provider_id + '_oauth_user_id'])
-                use_provider_user_id = True
-
-            if self.user_required and not u and not uid and not use_provider_user_id:
-                # raise ValueError("Cannot get OAuth token without an associated user")
-                return None
-            # check for user ID
-            if hasattr(self.model, "user_id") and uid:
-                query = query.filter_by(user_id=uid)
-            # check for user (relationship property)
-            elif hasattr(self.model, "user") and u:
-                query = query.filter_by(user=u)
-            # if we have the property, but not value, filter by None
-            elif hasattr(self.model, "user_id"):
-                query = query.filter_by(user_id=None)
-            # run query
-            try:
-                token = query.one().token
-            except NoResultFound:
-                token = None
-
-            # cache the result
-            self.cache.set(cache_key, token)
+    .. _SQLAlchemy: https://www.sqlalchemy.org/
+    """
+    def __init__(self, model, session, provider_id,
+                 user=None, user_id=None, user_required=None, anon_user=None,
+                 cache=None):
+        self.provider_id = provider_id
+        super(OAuthBackend, self).__init__(model, session, user, user_id, user_required, anon_user, cache)
 
+    def get(self, blueprint, user=None, user_id=None):
+        if self.provider_id + '_oauth_token' in session and session[self.provider_id + '_oauth_token'] != '':
+            return session[self.provider_id + '_oauth_token']
+        # check cache
+        cache_key = self.make_cache_key(blueprint=blueprint, user=user, user_id=user_id)
+        token = self.cache.get(cache_key)
+        if token:
             return token
 
-        def set(self, blueprint, token, user=None, user_id=None):
-            uid = first([user_id, self.user_id, blueprint.config.get("user_id")])
-            u = first(_get_real_user(ref, self.anon_user)
-                      for ref in (user, self.user, blueprint.config.get("user")))
+        # if not cached, make database queries
+        query = (
+            self.session.query(self.model)
+            .filter_by(provider=self.provider_id)
+        )
+        uid = first([user_id, self.user_id, blueprint.config.get("user_id")])
+        u = first(_get_real_user(ref, self.anon_user)
+                  for ref in (user, self.user, blueprint.config.get("user")))
 
-            if self.user_required and not u and not uid:
-                raise ValueError("Cannot set OAuth token without an associated user")
+        use_provider_user_id = False
+        if self.provider_id + '_oauth_user_id' in session and session[self.provider_id + '_oauth_user_id'] != '':
+            query = query.filter_by(provider_user_id=session[self.provider_id + '_oauth_user_id'])
+            use_provider_user_id = True
 
-            # if there was an existing model, delete it
-            existing_query = (
-                self.session.query(self.model)
-                .filter_by(provider=self.provider_id)
-            )
-            # check for user ID
-            has_user_id = hasattr(self.model, "user_id")
-            if has_user_id and uid:
-                existing_query = existing_query.filter_by(user_id=uid)
-            # check for user (relationship property)
-            has_user = hasattr(self.model, "user")
-            if has_user and u:
-                existing_query = existing_query.filter_by(user=u)
-            # queue up delete query -- won't be run until commit()
-            existing_query.delete()
-            # create a new model for this token
-            kwargs = {
-                "provider": self.provider_id,
-                "token": token,
-            }
-            if has_user_id and uid:
-                kwargs["user_id"] = uid
-            if has_user and u:
-                kwargs["user"] = u
-            self.session.add(self.model(**kwargs))
-            # commit to delete and add simultaneously
-            self.session.commit()
-            # invalidate cache
-            self.cache.delete(self.make_cache_key(
-                blueprint=blueprint, user=user, user_id=user_id
-            ))
+        if self.user_required and not u and not uid and not use_provider_user_id:
+            # raise ValueError("Cannot get OAuth token without an associated user")
+            return None
+        # check for user ID
+        if hasattr(self.model, "user_id") and uid:
+            query = query.filter_by(user_id=uid)
+        # check for user (relationship property)
+        elif hasattr(self.model, "user") and u:
+            query = query.filter_by(user=u)
+        # if we have the property, but not value, filter by None
+        elif hasattr(self.model, "user_id"):
+            query = query.filter_by(user_id=None)
+        # run query
+        try:
+            token = query.one().token
+        except NoResultFound:
+            token = None
 
-        def delete(self, blueprint, user=None, user_id=None):
-            query = (
-                self.session.query(self.model)
-                .filter_by(provider=self.provider_id)
-            )
-            uid = first([user_id, self.user_id, blueprint.config.get("user_id")])
-            u = first(_get_real_user(ref, self.anon_user)
-                      for ref in (user, self.user, blueprint.config.get("user")))
+        # cache the result
+        self.cache.set(cache_key, token)
 
-            if self.user_required and not u and not uid:
-                raise ValueError("Cannot delete OAuth token without an associated user")
+        return token
 
-            # check for user ID
-            if hasattr(self.model, "user_id") and uid:
-                query = query.filter_by(user_id=uid)
-            # check for user (relationship property)
-            elif hasattr(self.model, "user") and u:
-                query = query.filter_by(user=u)
-            # if we have the property, but not value, filter by None
-            elif hasattr(self.model, "user_id"):
-                query = query.filter_by(user_id=None)
-            # run query
-            query.delete()
-            self.session.commit()
-            # invalidate cache
-            self.cache.delete(self.make_cache_key(
-                blueprint=blueprint, user=user, user_id=user_id,
-            ))
+    def set(self, blueprint, token, user=None, user_id=None):
+        uid = first([user_id, self.user_id, blueprint.config.get("user_id")])
+        u = first(_get_real_user(ref, self.anon_user)
+                  for ref in (user, self.user, blueprint.config.get("user")))
 
-except Exception:
-    pass
+        if self.user_required and not u and not uid:
+            raise ValueError("Cannot set OAuth token without an associated user")
+
+        # if there was an existing model, delete it
+        existing_query = (
+            self.session.query(self.model)
+            .filter_by(provider=self.provider_id)
+        )
+        # check for user ID
+        has_user_id = hasattr(self.model, "user_id")
+        if has_user_id and uid:
+            existing_query = existing_query.filter_by(user_id=uid)
+        # check for user (relationship property)
+        has_user = hasattr(self.model, "user")
+        if has_user and u:
+            existing_query = existing_query.filter_by(user=u)
+        # queue up delete query -- won't be run until commit()
+        existing_query.delete()
+        # create a new model for this token
+        kwargs = {
+            "provider": self.provider_id,
+            "token": token,
+        }
+        if has_user_id and uid:
+            kwargs["user_id"] = uid
+        if has_user and u:
+            kwargs["user"] = u
+        self.session.add(self.model(**kwargs))
+        # commit to delete and add simultaneously
+        self.session.commit()
+        # invalidate cache
+        self.cache.delete(self.make_cache_key(
+            blueprint=blueprint, user=user, user_id=user_id
+        ))
+
+    def delete(self, blueprint, user=None, user_id=None):
+        query = (
+            self.session.query(self.model)
+            .filter_by(provider=self.provider_id)
+        )
+        uid = first([user_id, self.user_id, blueprint.config.get("user_id")])
+        u = first(_get_real_user(ref, self.anon_user)
+                  for ref in (user, self.user, blueprint.config.get("user")))
+
+        if self.user_required and not u and not uid:
+            raise ValueError("Cannot delete OAuth token without an associated user")
+
+        # check for user ID
+        if hasattr(self.model, "user_id") and uid:
+            query = query.filter_by(user_id=uid)
+        # check for user (relationship property)
+        elif hasattr(self.model, "user") and u:
+            query = query.filter_by(user=u)
+        # if we have the property, but not value, filter by None
+        elif hasattr(self.model, "user_id"):
+            query = query.filter_by(user_id=None)
+        # run query
+        query.delete()
+        self.session.commit()
+        # invalidate cache
+        self.cache.delete(self.make_cache_key(
+            blueprint=blueprint, user=user, user_id=user_id,
+        ))

cps/oauth_bb.py

@@ -30,12 +30,15 @@ from flask_babel import gettext as _
 from flask_dance.consumer import oauth_authorized, oauth_error
 from flask_dance.contrib.github import make_github_blueprint, github
 from flask_dance.contrib.google import make_google_blueprint, google
-from flask_login import login_user, current_user
+from flask_login import login_user, current_user, login_required
 from sqlalchemy.orm.exc import NoResultFound
 
 from . import constants, logger, config, app, ub
-from .web import login_required
-from .oauth import OAuthBackend, backend_resultcode
+
+try:
+    from .oauth import OAuthBackend, backend_resultcode
+except NameError:
+    pass
 
 
 oauth_check = {}
@@ -84,11 +87,7 @@ def register_user_with_oauth(user=None):
             except NoResultFound:
                 # no found, return error
                 return
-            try:
-                ub.session.commit()
-            except Exception as e:
-                log.exception(e)
-                ub.session.rollback()
+            ub.session_commit("User {} with OAuth for provider {} registered".format(user.nickname, oauth_key))
 
 
 def logout_oauth_user():
@@ -100,16 +99,12 @@ def logout_oauth_user():
 if ub.oauth_support:
     oauthblueprints = []
     if not ub.session.query(ub.OAuthProvider).count():
-        oauthProvider = ub.OAuthProvider()
-        oauthProvider.provider_name = "github"
-        oauthProvider.active = False
-        ub.session.add(oauthProvider)
-        ub.session.commit()
-        oauthProvider = ub.OAuthProvider()
-        oauthProvider.provider_name = "google"
-        oauthProvider.active = False
-        ub.session.add(oauthProvider)
-        ub.session.commit()
+        for provider in ("github", "google"):
+            oauthProvider = ub.OAuthProvider()
+            oauthProvider.provider_name = provider
+            oauthProvider.active = False
+            ub.session.add(oauthProvider)
+            ub.session_commit("{} Blueprint Created".format(provider))
 
     oauth_ids = ub.session.query(ub.OAuthProvider).all()
     ele1 = dict(provider_name='github',
@@ -199,12 +194,8 @@ if ub.oauth_support:
                 provider_user_id=provider_user_id,
                 token=token,
             )
-        try:
-            ub.session.add(oauth_entry)
-            ub.session.commit()
-        except Exception as e:
-            log.exception(e)
-            ub.session.rollback()
+        ub.session.add(oauth_entry)
+        ub.session_commit()
 
         # Disable Flask-Dance's default behavior for saving the OAuth token
         # Value differrs depending on flask-dance version
@@ -235,7 +226,7 @@ if ub.oauth_support:
                         flash(_(u"Link to %(oauth)s Succeeded", oauth=provider_name), category="success")
                         return redirect(url_for('web.profile'))
                     except Exception as e:
-                        log.exception(e)
+                        log.debug_or_exception(e)
                         ub.session.rollback()
                 else:
                     flash(_(u"Login failed, No User Linked With OAuth Account"), category="error")
@@ -282,7 +273,7 @@ if ub.oauth_support:
                     logout_oauth_user()
                     flash(_(u"Unlink to %(oauth)s Succeeded", oauth=oauth_check[provider]), category="success")
                 except Exception as e:
-                    log.exception(e)
+                    log.debug_or_exception(e)
                     ub.session.rollback()
                     flash(_(u"Unlink to %(oauth)s Failed", oauth=oauth_check[provider]), category="error")
         except NoResultFound:

cps/opds.py

@@ -25,7 +25,7 @@ import sys
 import datetime
 from functools import wraps
 
-from flask import Blueprint, request, render_template, Response, g, make_response
+from flask import Blueprint, request, render_template, Response, g, make_response, abort
 from flask_login import current_user
 from sqlalchemy.sql.expression import func, text, or_, and_
 from werkzeug.security import check_password_hash
@@ -33,7 +33,8 @@ from werkzeug.security import check_password_hash
 from . import constants, logger, config, db, calibre_db, ub, services, get_locale, isoLanguages
 from .helper import get_download_link, get_book_cover
 from .pagination import Pagination
-from .web import render_read_books, download_required
+from .web import render_read_books
+from .usermanagement import load_user_from_request
 from flask_babel import gettext as _
 from babel import Locale as LC
 from babel.core import UnknownLocaleError
@@ -51,7 +52,7 @@ def requires_basic_auth_if_no_ano(f):
             if not auth or auth.type != 'basic' or not check_auth(auth.username, auth.password):
                 return authenticate()
         return f(*args, **kwargs)
-    if config.config_login_type == constants.LOGIN_LDAP and services.ldap:
+    if config.config_login_type == constants.LOGIN_LDAP and services.ldap and config.config_anonbrowse != 1:
         return services.ldap.basic_auth_required(f)
     return decorated
 
@@ -100,7 +101,7 @@ def feed_normal_search():
 @requires_basic_auth_if_no_ano
 def feed_new():
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books, True, [db.Books.timestamp.desc()])
     return render_xml_template('feed.xml', entries=entries, pagination=pagination)
 
@@ -118,7 +119,7 @@ def feed_discover():
 @requires_basic_auth_if_no_ano
 def feed_best_rated():
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books, db.Books.ratings.any(db.Ratings.rating > 9),
                                                         [db.Books.timestamp.desc()])
     return render_xml_template('feed.xml', entries=entries, pagination=pagination)
@@ -164,7 +165,7 @@ def feed_authorindex():
 @requires_basic_auth_if_no_ano
 def feed_author(book_id):
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books,
                                                         db.Books.authors.any(db.Authors.id == book_id),
                                                         [db.Books.timestamp.desc()])
@@ -190,7 +191,7 @@ def feed_publisherindex():
 @requires_basic_auth_if_no_ano
 def feed_publisher(book_id):
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books,
                                                         db.Books.publishers.any(db.Publishers.id == book_id),
                                                         [db.Books.timestamp.desc()])
@@ -218,7 +219,7 @@ def feed_categoryindex():
 @requires_basic_auth_if_no_ano
 def feed_category(book_id):
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books,
                                                         db.Books.tags.any(db.Tags.id == book_id),
                                                         [db.Books.timestamp.desc()])
@@ -245,7 +246,7 @@ def feed_seriesindex():
 @requires_basic_auth_if_no_ano
 def feed_series(book_id):
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books,
                                                         db.Books.series.any(db.Series.id == book_id),
                                                         [db.Books.series_index])
@@ -276,7 +277,7 @@ def feed_ratingindex():
 @requires_basic_auth_if_no_ano
 def feed_ratings(book_id):
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books,
                                                         db.Books.ratings.any(db.Ratings.id == book_id),
                                                         [db.Books.timestamp.desc()])
@@ -304,7 +305,7 @@ def feed_formatindex():
 @requires_basic_auth_if_no_ano
 def feed_format(book_id):
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books,
                                                         db.Books.data.any(db.Data.format == book_id.upper()),
                                                         [db.Books.timestamp.desc()])
@@ -338,7 +339,7 @@ def feed_languagesindex():
 @requires_basic_auth_if_no_ano
 def feed_languages(book_id):
     off = request.args.get("offset") or 0
-    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1),
+    entries, __, pagination = calibre_db.fill_indexpage((int(off) / (int(config.config_books_per_page)) + 1), 0,
                                                         db.Books,
                                                         db.Books.languages.any(db.Languages.id == book_id),
                                                         [db.Books.timestamp.desc()])
@@ -383,8 +384,13 @@ def feed_shelf(book_id):
 
 @opds.route("/opds/download/<book_id>/<book_format>/")
 @requires_basic_auth_if_no_ano
-@download_required
 def opds_download_link(book_id, book_format):
+    # I gave up with this: With enabled ldap login, the user doesn't get logged in, therefore it's always guest
+    # workaround, loading the user from the request and checking it's download rights here
+    # in case of anonymous browsing user is None
+    user = load_user_from_request(request) or current_user
+    if not user.role_download():
+        return abort(403)
     if "Kobo" in request.headers.get('User-Agent'):
         client = "kobo"
     else:
@@ -408,7 +414,7 @@ def get_metadata_calibre_companion(uuid, library):
 
 def feed_search(term):
     if term:
-        entries = calibre_db.get_search_results(term)
+        entries, __, ___ = calibre_db.get_search_results(term)
         entriescount = len(entries) if len(entries) > 0 else 1
         pagination = Pagination(1, entriescount, entriescount)
         return render_xml_template('feed.xml', searchterm=term, entries=entries, pagination=pagination)
@@ -418,10 +424,18 @@ def feed_search(term):
 
 def check_auth(username, password):
     if sys.version_info.major == 3:
-        username = username.encode('windows-1252')
+        try:
+            username = username.encode('windows-1252')
+        except UnicodeEncodeError:
+            username = username.encode('utf-8')
     user = ub.session.query(ub.User).filter(func.lower(ub.User.nickname) ==
                                             username.decode('utf-8').lower()).first()
-    return bool(user and check_password_hash(str(user.password), password))
+    if bool(user and check_password_hash(str(user.password), password)):
+        return True
+    else:
+        ipAdress = request.headers.get('X-Forwarded-For', request.remote_addr)
+        log.warning('OPDS Login failed for user "%s" IP-address: %s', username.decode('utf-8'), ipAdress)
+        return False
 
 
 def authenticate():

cps/remotelogin.py

@@ -0,0 +1,138 @@
+# -*- coding: utf-8 -*-
+
+#  This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#    Copyright (C) 2018-2019 OzzieIsaacs, cervinko, jkrehm, bodybybuddha, ok11,
+#                            andy29485, idalin, Kyosfonica, wuqi, Kennyl, lemmsh,
+#                            falgh1, grunjol, csitko, ytils, xybydy, trasba, vrabe,
+#                            ruben-herold, marblepebble, JackED42, SiphonSquirrel,
+#                            apetresc, nanu-c, mutschler
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+import json
+from datetime import datetime
+
+from flask import Blueprint, request, make_response, abort, url_for, flash, redirect
+from flask_login import login_required, current_user, login_user
+from flask_babel import gettext as _
+from sqlalchemy.sql.expression import true
+
+from . import config, logger, ub
+from .render_template import render_title_template
+
+try:
+    from functools import wraps
+except ImportError:
+    pass  # We're not using Python 3
+
+remotelogin = Blueprint('remotelogin', __name__)
+log = logger.create()
+
+
+def remote_login_required(f):
+    @wraps(f)
+    def inner(*args, **kwargs):
+        if config.config_remote_login:
+            return f(*args, **kwargs)
+        if request.headers.get('X-Requested-With') == 'XMLHttpRequest':
+            data = {'status': 'error', 'message': 'Forbidden'}
+            response = make_response(json.dumps(data, ensure_ascii=False))
+            response.headers["Content-Type"] = "application/json; charset=utf-8"
+            return response, 403
+        abort(403)
+
+    return inner
+
+@remotelogin.route('/remote/login')
+@remote_login_required
+def remote_login():
+    auth_token = ub.RemoteAuthToken()
+    ub.session.add(auth_token)
+    ub.session_commit()
+    verify_url = url_for('remotelogin.verify_token', token=auth_token.auth_token, _external=true)
+    log.debug(u"Remot Login request with token: %s", auth_token.auth_token)
+    return render_title_template('remote_login.html', title=_(u"login"), token=auth_token.auth_token,
+                                 verify_url=verify_url, page="remotelogin")
+
+
+@remotelogin.route('/verify/<token>')
+@remote_login_required
+@login_required
+def verify_token(token):
+    auth_token = ub.session.query(ub.RemoteAuthToken).filter(ub.RemoteAuthToken.auth_token == token).first()
+
+    # Token not found
+    if auth_token is None:
+        flash(_(u"Token not found"), category="error")
+        log.error(u"Remote Login token not found")
+        return redirect(url_for('web.index'))
+
+    # Token expired
+    elif datetime.now() > auth_token.expiration:
+        ub.session.delete(auth_token)
+        ub.session_commit()
+
+        flash(_(u"Token has expired"), category="error")
+        log.error(u"Remote Login token expired")
+        return redirect(url_for('web.index'))
+
+    # Update token with user information
+    auth_token.user_id = current_user.id
+    auth_token.verified = True
+    ub.session_commit()
+
+    flash(_(u"Success! Please return to your device"), category="success")
+    log.debug(u"Remote Login token for userid %s verified", auth_token.user_id)
+    return redirect(url_for('web.index'))
+
+
+@remotelogin.route('/ajax/verify_token', methods=['POST'])
+@remote_login_required
+def token_verified():
+    token = request.form['token']
+    auth_token = ub.session.query(ub.RemoteAuthToken).filter(ub.RemoteAuthToken.auth_token == token).first()
+
+    data = {}
+
+    # Token not found
+    if auth_token is None:
+        data['status'] = 'error'
+        data['message'] = _(u"Token not found")
+
+    # Token expired
+    elif datetime.now() > auth_token.expiration:
+        ub.session.delete(auth_token)
+        ub.session_commit()
+
+        data['status'] = 'error'
+        data['message'] = _(u"Token has expired")
+
+    elif not auth_token.verified:
+        data['status'] = 'not_verified'
+
+    else:
+        user = ub.session.query(ub.User).filter(ub.User.id == auth_token.user_id).first()
+        login_user(user)
+
+        ub.session.delete(auth_token)
+        ub.session_commit("User {} logged in via remotelogin, token deleted".format(user.nickname))
+
+        data['status'] = 'success'
+        log.debug(u"Remote Login for userid %s succeded", user.id)
+        flash(_(u"you are now logged in as: '%(nickname)s'", nickname=user.nickname), category="success")
+
+    response = make_response(json.dumps(data, ensure_ascii=False))
+    response.headers["Content-Type"] = "application/json; charset=utf-8"
+
+    return response

cps/render_template.py

@@ -0,0 +1,116 @@
+# -*- coding: utf-8 -*-
+
+#  This file is part of the Calibre-Web (https://github.com/janeczku/calibre-web)
+#    Copyright (C) 2018-2020 OzzieIsaacs
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from flask import render_template
+from flask_babel import gettext as _
+from flask import g
+from werkzeug.local import LocalProxy
+from flask_login import current_user
+
+from . import config, constants, ub, logger, db, calibre_db
+from .ub import User
+
+
+log = logger.create()
+
+def get_sidebar_config(kwargs=None):
+    kwargs = kwargs or []
+    if 'content' in kwargs:
+        content = kwargs['content']
+        content = isinstance(content, (User, LocalProxy)) and not content.role_anonymous()
+    else:
+        content = 'conf' in kwargs
+    sidebar = list()
+    sidebar.append({"glyph": "glyphicon-book", "text": _('Books'), "link": 'web.index', "id": "new",
+                    "visibility": constants.SIDEBAR_RECENT, 'public': True, "page": "root",
+                    "show_text": _('Show recent books'), "config_show":False})
+    sidebar.append({"glyph": "glyphicon-fire", "text": _('Hot Books'), "link": 'web.books_list', "id": "hot",
+                    "visibility": constants.SIDEBAR_HOT, 'public': True, "page": "hot",
+                    "show_text": _('Show Hot Books'), "config_show": True})
+    sidebar.append({"glyph": "glyphicon-download", "text": _('Downloaded Books'), "link": 'web.books_list',
+                    "id": "download", "visibility": constants.SIDEBAR_DOWNLOAD, 'public': (not g.user.is_anonymous),
+                    "page": "download", "show_text": _('Show Downloaded Books'),
+                    "config_show": content})
+    sidebar.append(
+        {"glyph": "glyphicon-star", "text": _('Top Rated Books'), "link": 'web.books_list', "id": "rated",
+         "visibility": constants.SIDEBAR_BEST_RATED, 'public': True, "page": "rated",
+         "show_text": _('Show Top Rated Books'), "config_show": True})
+    sidebar.append({"glyph": "glyphicon-eye-open", "text": _('Read Books'), "link": 'web.books_list', "id": "read",
+                    "visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not g.user.is_anonymous),
+                    "page": "read", "show_text": _('Show read and unread'), "config_show": content})
+    sidebar.append(
+        {"glyph": "glyphicon-eye-close", "text": _('Unread Books'), "link": 'web.books_list', "id": "unread",
+         "visibility": constants.SIDEBAR_READ_AND_UNREAD, 'public': (not g.user.is_anonymous), "page": "unread",
+         "show_text": _('Show unread'), "config_show": False})
+    sidebar.append({"glyph": "glyphicon-random", "text": _('Discover'), "link": 'web.books_list', "id": "rand",
+                    "visibility": constants.SIDEBAR_RANDOM, 'public': True, "page": "discover",
+                    "show_text": _('Show random books'), "config_show": True})
+    sidebar.append({"glyph": "glyphicon-inbox", "text": _('Categories'), "link": 'web.category_list', "id": "cat",
+                    "visibility": constants.SIDEBAR_CATEGORY, 'public': True, "page": "category",
+                    "show_text": _('Show category selection'), "config_show": True})
+    sidebar.append({"glyph": "glyphicon-bookmark", "text": _('Series'), "link": 'web.series_list', "id": "serie",
+                    "visibility": constants.SIDEBAR_SERIES, 'public': True, "page": "series",
+                    "show_text": _('Show series selection'), "config_show": True})
+    sidebar.append({"glyph": "glyphicon-user", "text": _('Authors'), "link": 'web.author_list', "id": "author",
+                    "visibility": constants.SIDEBAR_AUTHOR, 'public': True, "page": "author",
+                    "show_text": _('Show author selection'), "config_show": True})
+    sidebar.append(
+        {"glyph": "glyphicon-text-size", "text": _('Publishers'), "link": 'web.publisher_list', "id": "publisher",
+         "visibility": constants.SIDEBAR_PUBLISHER, 'public': True, "page": "publisher",
+         "show_text": _('Show publisher selection'), "config_show":True})
+    sidebar.append({"glyph": "glyphicon-flag", "text": _('Languages'), "link": 'web.language_overview', "id": "lang",
+                    "visibility": constants.SIDEBAR_LANGUAGE, 'public': (g.user.filter_language() == 'all'),
+                    "page": "language",
+                    "show_text": _('Show language selection'), "config_show": True})
+    sidebar.append({"glyph": "glyphicon-star-empty", "text": _('Ratings'), "link": 'web.ratings_list', "id": "rate",
+                    "visibility": constants.SIDEBAR_RATING, 'public': True,
+                    "page": "rating", "show_text": _('Show ratings selection'), "config_show": True})
+    sidebar.append({"glyph": "glyphicon-file", "text": _('File formats'), "link": 'web.formats_list', "id": "format",
+                    "visibility": constants.SIDEBAR_FORMAT, 'public': True,
+                    "page": "format", "show_text": _('Show file formats selection'), "config_show": True})
+    sidebar.append(
+        {"glyph": "glyphicon-trash", "text": _('Archived Books'), "link": 'web.books_list', "id": "archived",
+         "visibility": constants.SIDEBAR_ARCHIVED, 'public': (not g.user.is_anonymous), "page": "archived",
+         "show_text": _('Show archived books'), "config_show": content})
+    sidebar.append(
+        {"glyph": "glyphicon-th-list", "text": _('Books List'), "link": 'web.books_table', "id": "list",
+         "visibility": constants.SIDEBAR_LIST, 'public': (not g.user.is_anonymous), "page": "list",
+         "show_text": _('Show Books List'), "config_show": content})
+
+    return sidebar
+
+def get_readbooks_ids():
+    if not config.config_read_column:
+        readBooks = ub.session.query(ub.ReadBook).filter(ub.ReadBook.user_id == int(current_user.id))\
+            .filter(ub.ReadBook.read_status == ub.ReadBook.STATUS_FINISHED).all()
+        return frozenset([x.book_id for x in readBooks])
+    else:
+        try:
+            readBooks = calibre_db.session.query(db.cc_classes[config.config_read_column])\
+                .filter(db.cc_classes[config.config_read_column].value == True).all()
+            return frozenset([x.book for x in readBooks])
+        except (KeyError, AttributeError):
+            log.error("Custom Column No.%d is not existing in calibre database", config.config_read_column)
+            return []
+
+# Returns the template for rendering and includes the instance name
+def render_title_template(*args, **kwargs):
+    sidebar = get_sidebar_config(kwargs)
+    return render_template(instance=config.config_calibre_web_title, sidebar=sidebar,
+                           accept=constants.EXTENSIONS_UPLOAD, read_book_ids=get_readbooks_ids(),
+                           *args, **kwargs)

cps/server.py

@@ -22,11 +22,14 @@ import os
 import errno
 import signal
 import socket
+import subprocess
 
 try:
     from gevent.pywsgi import WSGIServer
     from gevent.pool import Pool
     from gevent import __version__ as _version
+    from greenlet import GreenletExit
+    import ssl
     VERSION = 'Gevent ' + _version
     _GEVENT = True
 except ImportError:
@@ -134,6 +137,64 @@ class WebServer(object):
 
         return sock, _readable_listen_address(*address)
 
+    @staticmethod
+    def _get_args_for_reloading():
+        """Determine how the script was executed, and return the args needed
+        to execute it again in a new process.
+        Code from https://github.com/pyload/pyload. Author GammaC0de, voulter
+        """
+        rv = [sys.executable]
+        py_script = sys.argv[0]
+        args = sys.argv[1:]
+        # Need to look at main module to determine how it was executed.
+        __main__ = sys.modules["__main__"]
+
+        # The value of __package__ indicates how Python was called. It may
+        # not exist if a setuptools script is installed as an egg. It may be
+        # set incorrectly for entry points created with pip on Windows.
+        if getattr(__main__, "__package__", None) is None or (
+            os.name == "nt"
+            and __main__.__package__ == ""
+            and not os.path.exists(py_script)
+            and os.path.exists("{}.exe".format(py_script))
+        ):
+            # Executed a file, like "python app.py".
+            py_script = os.path.abspath(py_script)
+
+            if os.name == "nt":
+                # Windows entry points have ".exe" extension and should be
+                # called directly.
+                if not os.path.exists(py_script) and os.path.exists("{}.exe".format(py_script)):
+                    py_script += ".exe"
+
+                if (
+                        os.path.splitext(sys.executable)[1] == ".exe"
+                        and os.path.splitext(py_script)[1] == ".exe"
+                ):
+                    rv.pop(0)
+
+            rv.append(py_script)
+        else:
+            # Executed a module, like "python -m module".
+            if sys.argv[0] == "-m":
+                args = sys.argv
+            else:
+                if os.path.isfile(py_script):
+                    # Rewritten by Python from "-m script" to "/path/to/script.py".
+                    py_module = __main__.__package__
+                    name = os.path.splitext(os.path.basename(py_script))[0]
+
+                    if name != "__main__":
+                        py_module += ".{}".format(name)
+                else:
+                    # Incorrectly rewritten by pydevd debugger from "-m script" to "script".
+                    py_module = py_script
+
+                rv.extend(("-m", py_module.lstrip(".")))
+
+        rv.extend(args)
+        return rv
+
     def _start_gevent(self):
         ssl_args = self.ssl_args or {}
 
@@ -143,6 +204,16 @@ class WebServer(object):
                 output = _readable_listen_address(self.listen_address, self.listen_port)
             log.info('Starting Gevent server on %s', output)
             self.wsgiserver = WSGIServer(sock, self.app, log=self.access_logger, spawn=Pool(), **ssl_args)
+            if ssl_args:
+                wrap_socket = self.wsgiserver.wrap_socket
+                def my_wrap_socket(*args, **kwargs):
+                    try:
+                        return wrap_socket(*args, **kwargs)
+                    except (ssl.SSLError, OSError) as ex:
+                        log.warning('Gevent SSL Error: %s', ex)
+                        raise GreenletExit
+
+                self.wsgiserver.wrap_socket = my_wrap_socket
             self.wsgiserver.serve_forever()
         finally:
             if self.unix_socket_file:
@@ -187,22 +258,16 @@ class WebServer(object):
             return True
 
         log.info("Performing restart of Calibre-Web")
-        arguments = list(sys.argv)
-        arguments.insert(0, sys.executable)
-        if os.name == 'nt':
-            arguments = ["\"%s\"" % a for a in arguments]
-        os.execv(sys.executable, arguments)
+        args = self._get_args_for_reloading()
+        subprocess.call(args, close_fds=True)
         return True
 
-    def _killServer(self, ignored_signum, ignored_frame):
+    def _killServer(self, __, ___):
         self.stop()
 
     def stop(self, restart=False):
         from . import updater_thread
         updater_thread.stop()
-        from . import calibre_db
-        calibre_db.stop()
-
 
         log.info("webserver stop (restart=%s)", restart)
         self.restart = restart

cps/services/SyncToken.py

@@ -64,7 +64,7 @@ class SyncToken:
         books_last_modified: Datetime representing the last modified book that the device knows about.
     """
 
-    SYNC_TOKEN_HEADER = "x-kobo-synctoken"
+    SYNC_TOKEN_HEADER = "x-kobo-synctoken"  # nosec
     VERSION = "1-1-0"
     LAST_MODIFIED_ADDED_VERSION = "1-1-0"
     MIN_VERSION = "1-0-0"
@@ -85,17 +85,19 @@ class SyncToken:
             "archive_last_modified": {"type": "string"},
             "reading_state_last_modified": {"type": "string"},
             "tags_last_modified": {"type": "string"},
+            "books_last_id": {"type": "integer", "optional": True}
         },
     }
 
     def __init__(
         self,
-        raw_kobo_store_token="",
+        raw_kobo_store_token="",  # nosec
         books_last_created=datetime.min,
         books_last_modified=datetime.min,
         archive_last_modified=datetime.min,
         reading_state_last_modified=datetime.min,
         tags_last_modified=datetime.min,
+        books_last_id=-1
     ):
         self.raw_kobo_store_token = raw_kobo_store_token
         self.books_last_created = books_last_created
@@ -103,11 +105,12 @@ class SyncToken:
         self.archive_last_modified = archive_last_modified
         self.reading_state_last_modified = reading_state_last_modified
         self.tags_last_modified = tags_last_modified
+        self.books_last_id = books_last_id
 
     @staticmethod
     def from_headers(headers):
         sync_token_header = headers.get(SyncToken.SYNC_TOKEN_HEADER, "")
-        if sync_token_header == "":
+        if sync_token_header == "":  # nosec
             return SyncToken()
 
         # On the first sync from a Kobo device, we may receive the SyncToken
@@ -137,9 +140,12 @@ class SyncToken:
             archive_last_modified = get_datetime_from_json(data_json, "archive_last_modified")
             reading_state_last_modified = get_datetime_from_json(data_json, "reading_state_last_modified")
             tags_last_modified = get_datetime_from_json(data_json, "tags_last_modified")
+            books_last_id = data_json["books_last_id"]
         except TypeError:
             log.error("SyncToken timestamps don't parse to a datetime.")
             return SyncToken(raw_kobo_store_token=raw_kobo_store_token)
+        except KeyError:
+            books_last_id = -1
 
         return SyncToken(
             raw_kobo_store_token=raw_kobo_store_token,
@@ -147,7 +153,8 @@ class SyncToken:
             books_last_modified=books_last_modified,
             archive_last_modified=archive_last_modified,
             reading_state_last_modified=reading_state_last_modified,
-            tags_last_modified=tags_last_modified
+            tags_last_modified=tags_last_modified,
+            books_last_id=books_last_id
         )
 
     def set_kobo_store_header(self, store_headers):
@@ -170,7 +177,8 @@ class SyncToken:
                 "books_last_created": to_epoch_timestamp(self.books_last_created),
                 "archive_last_modified": to_epoch_timestamp(self.archive_last_modified),
                 "reading_state_last_modified": to_epoch_timestamp(self.reading_state_last_modified),
-                "tags_last_modified": to_epoch_timestamp(self.tags_last_modified)
+                "tags_last_modified": to_epoch_timestamp(self.tags_last_modified),
+                "books_last_id":self.books_last_id
             },
         }
         return b64encode_json(token)

cps/services/goodreads_support.py

@@ -72,7 +72,7 @@ def get_author_info(author_name):
         author_info = _client.find_author(author_name=author_name)
     except Exception as ex:
         # Skip goodreads, if site is down/inaccessible
-        log.warning('Goodreads website is down/inaccessible? %s', ex)
+        log.warning('Goodreads website is down/inaccessible? %s', ex.__str__())
         return
 
     if author_info:

cps/services/simpleldap.py

@@ -20,7 +20,7 @@ from __future__ import division, print_function, unicode_literals
 import base64
 
 from flask_simpleldap import LDAP, LDAPException
-
+from flask_simpleldap import ldap as pyLDAP
 from .. import constants, logger
 
 try:
@@ -38,6 +38,7 @@ def init_app(app, config):
 
     app.config['LDAP_HOST'] = config.config_ldap_provider_url
     app.config['LDAP_PORT'] = config.config_ldap_port
+    app.config['LDAP_CUSTOM_OPTIONS'] = {pyLDAP.OPT_REFERRALS: 0}
     if config.config_ldap_encryption == 2:
         app.config['LDAP_SCHEMA'] = 'ldaps'
     else:
@@ -54,8 +55,14 @@ def init_app(app, config):
         app.config['LDAP_USERNAME'] = ""
         app.config['LDAP_PASSWORD'] = base64.b64decode("")
     if bool(config.config_ldap_cert_path):
-        app.config['LDAP_REQUIRE_CERT'] = True
-        app.config['LDAP_CERT_PATH'] = config.config_ldap_cert_path
+        app.config['LDAP_CUSTOM_OPTIONS'].update({
+            pyLDAP.OPT_X_TLS_REQUIRE_CERT: pyLDAP.OPT_X_TLS_DEMAND,
+            pyLDAP.OPT_X_TLS_CACERTFILE: config.config_ldap_cacert_path,
+            pyLDAP.OPT_X_TLS_CERTFILE: config.config_ldap_cert_path,
+            pyLDAP.OPT_X_TLS_KEYFILE: config.config_ldap_key_path,
+            pyLDAP.OPT_X_TLS_NEWCTX: 0
+            })
+
     app.config['LDAP_BASE_DN'] = config.config_ldap_dn
     app.config['LDAP_USER_OBJECT_FILTER'] = config.config_ldap_user_object
 
@@ -67,12 +74,19 @@ def init_app(app, config):
 
     try:
         _ldap.init_app(app)
+    except ValueError:
+        if bool(config.config_ldap_cert_path):
+            app.config['LDAP_CUSTOM_OPTIONS'].pop(pyLDAP.OPT_X_TLS_NEWCTX)
+        try:
+            _ldap.init_app(app)
+        except RuntimeError as e:
+            log.error(e)
     except RuntimeError as e:
         log.error(e)
 
 
-def get_object_details(user=None, group=None, query_filter=None, dn_only=False):
-    return _ldap.get_object_details(user, group, query_filter, dn_only)
+def get_object_details(user=None,query_filter=None):
+    return _ldap.get_object_details(user, query_filter=query_filter)
 
 
 def bind():
@@ -103,7 +117,7 @@ def bind_user(username, password):
         return None, error
     except LDAPException as ex:
         if ex.message == 'Invalid credentials':
-            error = ("LDAP admin login failed")
+            error = "LDAP admin login failed"
             return None, error
         if ex.message == "Can't contact LDAP server":
             # log.warning('LDAP Server down: %s', ex)

cps/services/worker.py

@@ -0,0 +1,219 @@
+
+from __future__ import division, print_function, unicode_literals
+import threading
+import abc
+import uuid
+import time
+
+try:
+    import queue
+except ImportError:
+    import Queue as queue
+from datetime import datetime
+from collections import namedtuple
+
+from cps import logger
+
+log = logger.create()
+
+# task 'status' consts
+STAT_WAITING = 0
+STAT_FAIL = 1
+STAT_STARTED = 2
+STAT_FINISH_SUCCESS = 3
+
+# Only retain this many tasks in dequeued list
+TASK_CLEANUP_TRIGGER = 20
+
+QueuedTask = namedtuple('QueuedTask', 'num, user, added, task')
+
+
+def _get_main_thread():
+    for t in threading.enumerate():
+        if t.__class__.__name__ == '_MainThread':
+            return t
+    raise Exception("main thread not found?!")
+
+
+
+class ImprovedQueue(queue.Queue):
+    def to_list(self):
+        """
+        Returns a copy of all items in the queue without removing them.
+        """
+
+        with self.mutex:
+            return list(self.queue)
+
+#Class for all worker tasks in the background
+class WorkerThread(threading.Thread):
+    _instance = None
+
+    @classmethod
+    def getInstance(cls):
+        if cls._instance is None:
+            cls._instance = WorkerThread()
+        return cls._instance
+
+    def __init__(self):
+        threading.Thread.__init__(self)
+
+        self.dequeued = list()
+
+        self.doLock = threading.Lock()
+        self.queue = ImprovedQueue()
+        self.num = 0
+        self.start()
+
+    @classmethod
+    def add(cls, user, task):
+        ins = cls.getInstance()
+        ins.num += 1
+        ins.queue.put(QueuedTask(
+            num=ins.num,
+            user=user,
+            added=datetime.now(),
+            task=task,
+        ))
+
+    @property
+    def tasks(self):
+        with self.doLock:
+            tasks = self.queue.to_list() + self.dequeued
+            return sorted(tasks, key=lambda x: x.num)
+
+    def cleanup_tasks(self):
+        with self.doLock:
+            dead = []
+            alive = []
+            for x in self.dequeued:
+                (dead if x.task.dead else alive).append(x)
+
+            # if the ones that we need to keep are within the trigger, do nothing else
+            delta = len(self.dequeued) - len(dead)
+            if delta > TASK_CLEANUP_TRIGGER:
+                ret = alive
+            else:
+                # otherwise, lop off the oldest dead tasks until we hit the target trigger
+                ret = sorted(dead, key=lambda x: x.task.end_time)[-TASK_CLEANUP_TRIGGER:] + alive
+
+            self.dequeued = sorted(ret, key=lambda x: x.num)
+
+    # Main thread loop starting the different tasks
+    def run(self):
+        main_thread = _get_main_thread()
+        while main_thread.is_alive():
+            try:
+                # this blocks until something is available. This can cause issues when the main thread dies - this
+                # thread will remain alive. We implement a timeout to unblock every second which allows us to check if
+                # the main thread is still alive.
+                # We don't use a daemon here because we don't want the tasks to just be abruptly halted, leading to
+                # possible file / database corruption
+                item = self.queue.get(timeout=1)
+            except queue.Empty:
+                time.sleep(1)
+                continue
+
+            with self.doLock:
+                # add to list so that in-progress tasks show up
+                self.dequeued.append(item)
+
+            # once we hit our trigger, start cleaning up dead tasks
+            if len(self.dequeued) > TASK_CLEANUP_TRIGGER:
+                self.cleanup_tasks()
+
+            # sometimes tasks (like Upload) don't actually have work to do and are created as already finished
+            if item.task.stat is STAT_WAITING:
+                # CalibreTask.start() should wrap all exceptions in it's own error handling
+                item.task.start(self)
+
+            self.queue.task_done()
+
+
+class CalibreTask:
+    __metaclass__ = abc.ABCMeta
+
+    def __init__(self, message):
+        self._progress = 0
+        self.stat = STAT_WAITING
+        self.error = None
+        self.start_time = None
+        self.end_time = None
+        self.message = message
+        self.id = uuid.uuid4()
+
+    @abc.abstractmethod
+    def run(self, worker_thread):
+        """Provides the caller some human-readable name for this class"""
+        raise NotImplementedError
+
+    @abc.abstractmethod
+    def name(self):
+        """Provides the caller some human-readable name for this class"""
+        raise NotImplementedError
+
+    def start(self, *args):
+        self.start_time = datetime.now()
+        self.stat = STAT_STARTED
+
+        # catch any unhandled exceptions in a task and automatically fail it
+        try:
+            self.run(*args)
+        except Exception as e:
+            self._handleError(str(e))
+            log.debug_or_exception(e)
+
+        self.end_time = datetime.now()
+
+    @property
+    def stat(self):
+        return self._stat
+
+    @stat.setter
+    def stat(self, x):
+        self._stat = x
+
+    @property
+    def progress(self):
+        return self._progress
+
+    @progress.setter
+    def progress(self, x):
+        if not 0 <= x <= 1:
+            raise ValueError("Task progress should within [0, 1] range")
+        self._progress = x
+
+    @property
+    def error(self):
+        return self._error
+
+    @error.setter
+    def error(self, x):
+        self._error = x
+
+    @property
+    def runtime(self):
+        return (self.end_time or datetime.now()) - self.start_time
+
+    @property
+    def dead(self):
+        """Determines whether or not this task can be garbage collected
+
+        We have a separate dictating this because there may be certain tasks that want to override this
+        """
+        # By default, we're good to clean a task if it's "Done"
+        return self.stat in (STAT_FINISH_SUCCESS, STAT_FAIL)
+
+    @progress.setter
+    def progress(self, x):
+        # todo: throw error if outside of [0,1]
+        self._progress = x
+
+    def _handleError(self, error_message):
+        self.stat = STAT_FAIL
+        self.progress = 1
+        self.error = error_message
+
+    def _handleSuccess(self):
+        self.stat = STAT_FINISH_SUCCESS
+        self.progress = 1

cps/shelf.py

@@ -22,15 +22,17 @@
 
 from __future__ import division, print_function, unicode_literals
 from datetime import datetime
+import sys
 
 from flask import Blueprint, request, flash, redirect, url_for
 from flask_babel import gettext as _
 from flask_login import login_required, current_user
-from sqlalchemy.sql.expression import func
+from sqlalchemy.sql.expression import func, true
 from sqlalchemy.exc import OperationalError, InvalidRequestError
 
-from . import logger, ub, searched_ids, calibre_db
-from .web import login_required_if_no_ano, render_title_template
+from . import logger, ub, calibre_db, db
+from .render_template import render_title_template
+from .usermanagement import login_required_if_no_ano
 
 
 shelf = Blueprint('shelf', __name__)
@@ -124,32 +126,28 @@ def search_to_shelf(shelf_id):
         flash(_(u"You are not allowed to add a book to the the shelf: %(name)s", name=shelf.name), category="error")
         return redirect(url_for('web.index'))
 
-    if current_user.id in searched_ids and searched_ids[current_user.id]:
+    if current_user.id in ub.searched_ids and ub.searched_ids[current_user.id]:
         books_for_shelf = list()
         books_in_shelf = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).all()
         if books_in_shelf:
             book_ids = list()
             for book_id in books_in_shelf:
                 book_ids.append(book_id.book_id)
-            for searchid in searched_ids[current_user.id]:
+            for searchid in ub.searched_ids[current_user.id]:
                 if searchid not in book_ids:
                     books_for_shelf.append(searchid)
         else:
-            books_for_shelf = searched_ids[current_user.id]
+            books_for_shelf = ub.searched_ids[current_user.id]
 
         if not books_for_shelf:
-            log.error("Books are already part of %s", shelf)
+            log.error("Books are already part of %s", shelf.name)
             flash(_(u"Books are already part of the shelf: %(name)s", name=shelf.name), category="error")
             return redirect(url_for('web.index'))
 
-        maxOrder = ub.session.query(func.max(ub.BookShelf.order)).filter(ub.BookShelf.shelf == shelf_id).first()
-        if maxOrder[0] is None:
-            maxOrder = 0
-        else:
-            maxOrder = maxOrder[0]
+        maxOrder = ub.session.query(func.max(ub.BookShelf.order)).filter(ub.BookShelf.shelf == shelf_id).first()[0] or 0
 
         for book in books_for_shelf:
-            maxOrder = maxOrder + 1
+            maxOrder += 1
             shelf.books.append(ub.BookShelf(shelf=shelf.id, book_id=book, order=maxOrder))
         shelf.last_modified = datetime.utcnow()
         try:
@@ -223,96 +221,76 @@ def remove_from_shelf(shelf_id, book_id):
 @login_required
 def create_shelf():
     shelf = ub.Shelf()
-    if request.method == "POST":
-        to_save = request.form.to_dict()
-        if "is_public" in to_save:
-            shelf.is_public = 1
-        shelf.name = to_save["title"]
-        shelf.user_id = int(current_user.id)
-
-        is_shelf_name_unique = False
-        if shelf.is_public == 1:
-            is_shelf_name_unique = ub.session.query(ub.Shelf) \
-                .filter((ub.Shelf.name == to_save["title"]) & (ub.Shelf.is_public == 1)) \
-                .first() is None
-
-            if not is_shelf_name_unique:
-                flash(_(u"A public shelf with the name '%(title)s' already exists.", title=to_save["title"]),
-                      category="error")
-        else:
-            is_shelf_name_unique = ub.session.query(ub.Shelf) \
-                .filter((ub.Shelf.name == to_save["title"]) & (ub.Shelf.is_public == 0) &
-                        (ub.Shelf.user_id == int(current_user.id)))\
-                                       .first() is None
-
-            if not is_shelf_name_unique:
-                flash(_(u"A private shelf with the name '%(title)s' already exists.", title=to_save["title"]),
-                      category="error")
-
-        if is_shelf_name_unique:
-            try:
-                ub.session.add(shelf)
-                ub.session.commit()
-                flash(_(u"Shelf %(title)s created", title=to_save["title"]), category="success")
-                return redirect(url_for('shelf.show_shelf', shelf_id=shelf.id))
-            except (OperationalError, InvalidRequestError):
-                ub.session.rollback()
-                flash(_(u"Settings DB is not Writeable"), category="error")
-            except Exception:
-                ub.session.rollback()
-                flash(_(u"There was an error"), category="error")
-        return render_title_template('shelf_edit.html', shelf=shelf, title=_(u"Create a Shelf"), page="shelfcreate")
-    else:
-        return render_title_template('shelf_edit.html', shelf=shelf, title=_(u"Create a Shelf"), page="shelfcreate")
+    return create_edit_shelf(shelf, title=_(u"Create a Shelf"), page="shelfcreate")
 
 
 @shelf.route("/shelf/edit/<int:shelf_id>", methods=["GET", "POST"])
 @login_required
 def edit_shelf(shelf_id):
     shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
+    return create_edit_shelf(shelf, title=_(u"Edit a shelf"), page="shelfedit", shelf_id=shelf_id)
+
+
+# if shelf ID is set, we are editing a shelf
+def create_edit_shelf(shelf, title, page, shelf_id=False):
     if request.method == "POST":
         to_save = request.form.to_dict()
-
-        is_shelf_name_unique = False
-        if shelf.is_public == 1:
-            is_shelf_name_unique = ub.session.query(ub.Shelf) \
-                .filter((ub.Shelf.name == to_save["title"]) & (ub.Shelf.is_public == 1)) \
-                .filter(ub.Shelf.id != shelf_id) \
-                .first() is None
-
-            if not is_shelf_name_unique:
-                flash(_(u"A public shelf with the name '%(title)s' already exists.", title=to_save["title"]),
-                      category="error")
+        if "is_public" in to_save:
+            shelf.is_public = 1
         else:
-            is_shelf_name_unique = ub.session.query(ub.Shelf) \
-                .filter((ub.Shelf.name == to_save["title"]) & (ub.Shelf.is_public == 0) &
-                        (ub.Shelf.user_id == int(current_user.id)))\
-                                       .filter(ub.Shelf.id != shelf_id)\
-                                       .first() is None
-
-            if not is_shelf_name_unique:
-                flash(_(u"A private shelf with the name '%(title)s' already exists.", title=to_save["title"]),
-                      category="error")
-
-        if is_shelf_name_unique:
+            shelf.is_public = 0
+        if check_shelf_is_unique(shelf, to_save, shelf_id):
             shelf.name = to_save["title"]
-            shelf.last_modified = datetime.utcnow()
-            if "is_public" in to_save:
-                shelf.is_public = 1
+            # shelf.last_modified = datetime.utcnow()
+            if not shelf_id:
+                shelf.user_id = int(current_user.id)
+                ub.session.add(shelf)
+                shelf_action = "created"
+                flash_text = _(u"Shelf %(title)s created", title=to_save["title"])
             else:
-                shelf.is_public = 0
+                shelf_action = "changed"
+                flash_text = _(u"Shelf %(title)s changed", title=to_save["title"])
             try:
                 ub.session.commit()
-                flash(_(u"Shelf %(title)s changed", title=to_save["title"]), category="success")
-            except (OperationalError, InvalidRequestError):
+                log.info(u"Shelf {} {}".format(to_save["title"], shelf_action))
+                flash(flash_text, category="success")
+                return redirect(url_for('shelf.show_shelf', shelf_id=shelf.id))
+            except (OperationalError, InvalidRequestError) as e:
                 ub.session.rollback()
+                log.debug_or_exception(e)
                 flash(_(u"Settings DB is not Writeable"), category="error")
-            except Exception:
+            except Exception as e:
                 ub.session.rollback()
+                log.debug_or_exception(e)
                 flash(_(u"There was an error"), category="error")
-        return render_title_template('shelf_edit.html', shelf=shelf, title=_(u"Edit a shelf"), page="shelfedit")
+    return render_title_template('shelf_edit.html', shelf=shelf, title=title, page=page)
+
+
+def check_shelf_is_unique(shelf, to_save, shelf_id=False):
+    if shelf_id:
+        ident = ub.Shelf.id != shelf_id
     else:
-        return render_title_template('shelf_edit.html', shelf=shelf, title=_(u"Edit a shelf"), page="shelfedit")
+        ident = true()
+    if shelf.is_public == 1:
+        is_shelf_name_unique = ub.session.query(ub.Shelf) \
+                                   .filter((ub.Shelf.name == to_save["title"]) & (ub.Shelf.is_public == 1)) \
+                                   .filter(ident) \
+                                   .first() is None
+
+        if not is_shelf_name_unique:
+            flash(_(u"A public shelf with the name '%(title)s' already exists.", title=to_save["title"]),
+                  category="error")
+    else:
+        is_shelf_name_unique = ub.session.query(ub.Shelf) \
+                                   .filter((ub.Shelf.name == to_save["title"]) & (ub.Shelf.is_public == 0) &
+                                           (ub.Shelf.user_id == int(current_user.id))) \
+                                   .filter(ident) \
+                                   .first() is None
+
+        if not is_shelf_name_unique:
+            flash(_(u"A private shelf with the name '%(title)s' already exists.", title=to_save["title"]),
+                  category="error")
+    return is_shelf_name_unique
 
 
 def delete_shelf_helper(cur_shelf):
@@ -322,9 +300,7 @@ def delete_shelf_helper(cur_shelf):
     ub.session.delete(cur_shelf)
     ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id).delete()
     ub.session.add(ub.ShelfArchive(uuid=cur_shelf.uuid, user_id=cur_shelf.user_id))
-    ub.session.commit()
-    log.info("successfully deleted %s", cur_shelf)
-
+    ub.session_commit("successfully deleted Shelf {}".format(cur_shelf.name))
 
 
 @shelf.route("/shelf/delete/<int:shelf_id>")
@@ -333,44 +309,24 @@ def delete_shelf(shelf_id):
     cur_shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
     try:
         delete_shelf_helper(cur_shelf)
-    except (OperationalError, InvalidRequestError):
+    except InvalidRequestError:
         ub.session.rollback()
         flash(_(u"Settings DB is not Writeable"), category="error")
     return redirect(url_for('web.index'))
 
 
-@shelf.route("/shelf/<int:shelf_id>", defaults={'shelf_type': 1})
-@shelf.route("/shelf/<int:shelf_id>/<int:shelf_type>")
+@shelf.route("/simpleshelf/<int:shelf_id>")
 @login_required_if_no_ano
-def show_shelf(shelf_type, shelf_id):
-    shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
+def show_simpleshelf(shelf_id):
+    return render_show_shelf(2, shelf_id, 1, None)
 
-    result = list()
-    # user is allowed to access shelf
-    if shelf and check_shelf_view_permissions(shelf):
-        page = "shelf.html" if shelf_type == 1 else 'shelfdown.html'
 
-        books_in_shelf = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id)\
-            .order_by(ub.BookShelf.order.asc()).all()
-        for book in books_in_shelf:
-            cur_book = calibre_db.get_filtered_book(book.book_id)
-            if cur_book:
-                result.append(cur_book)
-            else:
-                cur_book = calibre_db.get_book(book.book_id)
-                if not cur_book:
-                    log.info('Not existing book %s in %s deleted', book.book_id, shelf)
-                    try:
-                        ub.session.query(ub.BookShelf).filter(ub.BookShelf.book_id == book.book_id).delete()
-                        ub.session.commit()
-                    except (OperationalError, InvalidRequestError):
-                        ub.session.rollback()
-                        flash(_(u"Settings DB is not Writeable"), category="error")
-        return render_title_template(page, entries=result, title=_(u"Shelf: '%(name)s'", name=shelf.name),
-                                     shelf=shelf, page="shelf")
-    else:
-        flash(_(u"Error opening shelf. Shelf does not exist or is not accessible"), category="error")
-        return redirect(url_for("web.index"))
+@shelf.route("/shelf/<int:shelf_id>", defaults={"sort_param": "order", 'page': 1})
+@shelf.route("/shelf/<int:shelf_id>/<sort_param>", defaults={'page': 1})
+@shelf.route("/shelf/<int:shelf_id>/<sort_param>/<int:page>")
+@login_required_if_no_ano
+def show_shelf(shelf_id, sort_param, page):
+    return render_show_shelf(1, shelf_id, page, sort_param)
 
 
 @shelf.route("/shelf/order/<int:shelf_id>", methods=["GET", "POST"])
@@ -394,22 +350,79 @@ def order_shelf(shelf_id):
     shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
     result = list()
     if shelf and check_shelf_view_permissions(shelf):
-        books_in_shelf2 = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id) \
-            .order_by(ub.BookShelf.order.asc()).all()
-        for book in books_in_shelf2:
-            cur_book = calibre_db.get_filtered_book(book.book_id)
-            if cur_book:
-                result.append({'title': cur_book.title,
-                               'id': cur_book.id,
-                               'author': cur_book.authors,
-                               'series': cur_book.series,
-                               'series_index': cur_book.series_index})
-            else:
-                cur_book = calibre_db.get_book(book.book_id)
-                result.append({'title': _('Hidden Book'),
-                               'id': cur_book.id,
-                               'author': [],
-                               'series': []})
+        result = calibre_db.session.query(db.Books)\
+            .join(ub.BookShelf,ub.BookShelf.book_id == db.Books.id , isouter=True) \
+            .add_columns(calibre_db.common_filters().label("visible")) \
+            .filter(ub.BookShelf.shelf == shelf_id).order_by(ub.BookShelf.order.asc()).all()
     return render_title_template('shelf_order.html', entries=result,
                                  title=_(u"Change order of Shelf: '%(name)s'", name=shelf.name),
                                  shelf=shelf, page="shelforder")
+
+
+def change_shelf_order(shelf_id, order):
+    result = calibre_db.session.query(db.Books).join(ub.BookShelf,ub.BookShelf.book_id == db.Books.id)\
+        .filter(ub.BookShelf.shelf == shelf_id).order_by(*order).all()
+    for index, entry in enumerate(result):
+        book = ub.session.query(ub.BookShelf).filter(ub.BookShelf.shelf == shelf_id) \
+            .filter(ub.BookShelf.book_id == entry.id).first()
+        book.order = index
+    ub.session_commit("Shelf-id:{} - Order changed".format(shelf_id))
+
+
+def render_show_shelf(shelf_type, shelf_id, page_no, sort_param):
+    shelf = ub.session.query(ub.Shelf).filter(ub.Shelf.id == shelf_id).first()
+
+    # check user is allowed to access shelf
+    if shelf and check_shelf_view_permissions(shelf):
+
+        if shelf_type == 1:
+            # order = [ub.BookShelf.order.asc()]
+            if sort_param == 'pubnew':
+                change_shelf_order(shelf_id, [db.Books.pubdate.desc()])
+            if sort_param == 'pubold':
+                change_shelf_order(shelf_id, [db.Books.pubdate])
+            if sort_param == 'abc':
+                change_shelf_order(shelf_id, [db.Books.sort])
+            if sort_param == 'zyx':
+                change_shelf_order(shelf_id, [db.Books.sort.desc()])
+            if sort_param == 'new':
+                change_shelf_order(shelf_id, [db.Books.timestamp.desc()])
+            if sort_param == 'old':
+                change_shelf_order(shelf_id, [db.Books.timestamp])
+            if sort_param == 'authaz':
+                change_shelf_order(shelf_id, [db.Books.author_sort.asc()])
+            if sort_param == 'authza':
+                change_shelf_order(shelf_id, [db.Books.author_sort.desc()])
+            page = "shelf.html"
+            pagesize = 0
+        else:
+            pagesize = sys.maxsize
+            page = 'shelfdown.html'
+
+        result, __, pagination = calibre_db.fill_indexpage(page_no, pagesize,
+                                                            db.Books,
+                                                            ub.BookShelf.shelf == shelf_id,
+                                                            [ub.BookShelf.order.asc()],
+                                                            ub.BookShelf,ub.BookShelf.book_id == db.Books.id)
+        # delete chelf entries where book is not existent anymore, can happen if book is deleted outside calibre-web
+        wrong_entries = calibre_db.session.query(ub.BookShelf)\
+            .join(db.Books, ub.BookShelf.book_id == db.Books.id, isouter=True)\
+            .filter(db.Books.id == None).all()
+        for entry in wrong_entries:
+            log.info('Not existing book {} in {} deleted'.format(entry.book_id, shelf))
+            try:
+                ub.session.query(ub.BookShelf).filter(ub.BookShelf.book_id == entry.book_id).delete()
+                ub.session.commit()
+            except (OperationalError, InvalidRequestError):
+                ub.session.rollback()
+                flash(_(u"Settings DB is not Writeable"), category="error")
+
+        return render_title_template(page,
+                                     entries=result,
+                                     pagination=pagination,
+                                     title=_(u"Shelf: '%(name)s'", name=shelf.name),
+                                     shelf=shelf,
+                                     page="shelf")
+    else:
+        flash(_(u"Error opening shelf. Shelf does not exist or is not accessible"), category="error")
+        return redirect(url_for("web.index"))

cps/static/css/caliBlur_override.css

@@ -1,17 +1,24 @@
-body.serieslist.grid-view div.container-fluid>div>div.col-sm-10:before{
-    display: none;
+body.serieslist.grid-view div.container-fluid > div > div.col-sm-10::before {
+  display: none;
 }
 
-.cover .badge{
-    position: absolute;
-    top: 0;
-    right: 0;
-    background-color: #cc7b19;
-    border-radius: 0;
-    padding: 0 8px;
-    box-shadow: 0 0 4px rgba(0,0,0,.6);
-    line-height: 24px;
+.cover .badge {
+  position: absolute;
+  top: 0;
+  left: 0;
+  color: #fff;
+  background-color: #cc7b19;
+  border-radius: 0;
+  padding: 0 8px;
+  box-shadow: 0 0 4px rgba(0, 0, 0, 0.6);
+  line-height: 24px;
 }
-.cover{
-    box-shadow: 0 0 4px rgba(0,0,0,.6);
+
+.cover {
+  box-shadow: 0 0 4px rgba(0, 0, 0, 0.6);
+}
+
+.cover .read {
+  padding: 0 0;
+  line-height: 15px;
 }

cps/static/css/kthoom.css

@@ -33,7 +33,6 @@ body {
   position: relative;
   cursor: pointer;
   padding: 4px;
-
   transition: all 0.2s ease;
 }
 
@@ -45,7 +44,7 @@ body {
 
 #sidebar a.active,
 #sidebar a.active img + span {
-  background-color: #45B29D;
+  background-color: #45b29d;
 }
 
 #sidebar li img {
@@ -99,7 +98,7 @@ body {
   background-color: #ccc;
 }
 
-#progress .bar-read  {
+#progress .bar-read {
   color: #fff;
   background-color: #45b29d;
 }

cps/static/css/listen.css

@@ -66,19 +66,12 @@ body {
   right: 40px;
 }
 
-xmp,
-pre,
-plaintext {
-  display: block;
-  font-family: -moz-fixed;
-  white-space: pre;
-  margin: 1em 0;
-}
-
 pre {
+  display: block;
+  margin: 1em 0;
   white-space: pre-wrap;
   word-wrap: break-word;
-  font-family: -moz-fixed;
+  font-family: -moz-fixed, sans-serif;
   column-count: 2;
   -webkit-columns: 2;
   -moz-columns: 2;

cps/static/css/main.css

@@ -25,10 +25,9 @@ body {
   overflow: hidden;
   -webkit-transition: -webkit-transform 0.4s, width 0.2s;
   -moz-transition: -webkit-transform 0.4s, width 0.2s;
-  -ms-transition: -webkit-transform 0.4s, width 0.2s;
+  transition: -webkit-transform 0.4s, width 0.2s;
   -moz-box-shadow: inset 0 0 50px rgba(0, 0, 0, 0.1);
   -webkit-box-shadow: inset 0 0 50px rgba(0, 0, 0, 0.1);
-  -ms-box-shadow: inset 0 0 50px rgba(0, 0, 0, 0.1);
   box-shadow: inset 0 0 50px rgba(0, 0, 0, 0.1);
 }
 
@@ -36,7 +35,6 @@ body {
   height: 8%;
   min-height: 20px;
   padding: 10px;
-  /* margin: 0 50px 0 50px; */
   position: relative;
   color: #4f4f4f;
   font-weight: 100;
@@ -45,7 +43,7 @@ body {
   text-align: center;
   -webkit-transition: opacity 0.5s;
   -moz-transition: opacity 0.5s;
-  -ms-transition: opacity 0.5s;
+  transition: opacity 0.5s;
   z-index: 10;
 }
 
@@ -74,12 +72,21 @@ body {
   padding: 3px;
 }
 
+#panels a {
+  visibility: hidden;
+  width: 18px;
+  height: 20px;
+  overflow: hidden;
+  display: inline-block;
+  color: #ccc;
+  margin-left: 6px;
+}
+
 #titlebar a:active {
   opacity: 1;
   color: rgba(0, 0, 0, 0.6);
   -moz-box-shadow: inset 0 0 6px rgba(155, 155, 155, 0.8);
   -webkit-box-shadow: inset 0 0 6px rgba(155, 155, 155, 0.8);
-  -ms-box-shadow: inset 0 0 6px rgba(155, 155, 155, 0.8);
   box-shadow: inset 0 0 6px rgba(155, 155, 155, 0.8);
 }
 
@@ -116,12 +123,11 @@ body {
   top: 50%;
   margin-top: -192px;
   font-size: 64px;
-  color: #E2E2E2;
+  color: #e2e2e2;
   font-family: arial, sans-serif;
   font-weight: bold;
   cursor: pointer;
   -webkit-user-select: none;
-  -khtml-user-select: none;
   -moz-user-select: none;
   -ms-user-select: none;
   user-select: none;
@@ -147,16 +153,10 @@ body {
   height: 100%;
   -webkit-transition: -webkit-transform 0.5s;
   -moz-transition: -moz-transform 0.5s;
-  -ms-transition: -moz-transform 0.5s;
+  transition: -moz-transform 0.5s;
   overflow: hidden;
 }
 
-#sidebar.open {
-  /* left: 0; */
-  /* -webkit-transform: translate(0, 0);
-  -moz-transform: translate(0, 0); */
-}
-
 #main.closed {
   /* left: 300px; */
   -webkit-transform: translate(300px, 0);
@@ -183,7 +183,6 @@ body {
   height: 14px;
   -moz-box-shadow: 0 1px 3px rgba(0, 0, 0, 0.6);
   -webkit-box-shadow: 0 1px 3px rgba(0, 0, 0, 0.6);
-  -ms-box-shadow: 0 1px 3px rgba(0, 0, 0, 0.6);
   box-shadow: 0 1px 3px rgba(0, 0, 0, 0.6);
 }
 
@@ -200,19 +199,24 @@ body {
 
 #title-controls { float: right; }
 
-#panels a {
-  visibility: hidden;
-  width: 18px;
-  height: 20px;
-  overflow: hidden;
-  display: inline-block;
-  color: #ccc;
-  margin-left: 6px;
-}
-
 #panels a::before { visibility: visible; }
 #panels a:hover { color: #aaa; }
 
+.list_item.currentChapter > a,
+.list_item a:hover {
+  color: #f1f1f1;
+}
+
+.list_item a {
+  color: #aaa;
+  text-decoration: none;
+}
+
+#searchResults a {
+  color: #aaa;
+  text-decoration: none;
+}
+
 #panels a:active {
   color: #aaa;
   margin: 1px 0 -1px 6px;
@@ -232,7 +236,6 @@ body {
 
 input::-webkit-input-placeholder { color: #454545; }
 input:-moz-placeholder { color: #454545; }
-input:-ms-placeholder { color: #454545; }
 
 #divider {
   position: absolute;
@@ -243,7 +246,7 @@ input:-ms-placeholder { color: #454545; }
   left: 50%;
   margin-left: -1px;
   top: 10%;
-  opacity: .15;
+  opacity: 0.15;
   box-shadow: -2px 0 15px rgba(0, 0, 0, 1);
   display: none;
 }
@@ -268,24 +271,35 @@ input:-ms-placeholder { color: #454545; }
   width: 25%;
   height: 100%;
   visibility: hidden;
-  -webkit-transition: visibility 0 ease 0.5s;
-  -moz-transition: visibility 0 ease 0.5s;
-  -ms-transition: visibility 0 ease 0.5s;
+  -webkit-transition: visibility 0s ease 0.5s;
+  -moz-transition: visibility 0s ease 0.5s;
+  transition: visibility 0s ease 0.5s;
 }
 
 #sidebar.open #tocView,
 #sidebar.open #bookmarksView {
   overflow-y: auto;
   visibility: visible;
-  -webkit-transition: visibility 0 ease 0;
-  -moz-transition: visibility 0 ease 0;
-  -ms-transition: visibility 0 ease 0;
+  -webkit-transition: visibility 0s ease 0s;
+  -moz-transition: visibility 0s ease 0s;
+  transition: visibility 0s ease 0s;
 }
 
 #sidebar.open #tocView {
   display: block;
 }
 
+.list_item ul {
+  padding-left: 10px;
+  margin-top: 8px;
+  display: none;
+}
+
+.list_item.currentChapter > ul,
+.list_item.openChapter > ul {
+  display: block;
+}
+
 #tocView > ul,
 #bookmarksView > ul {
   margin-top: 15px;
@@ -296,22 +310,41 @@ input:-ms-placeholder { color: #454545; }
 
 #tocView li,
 #bookmarksView li {
-  margin-bottom:10px;
+  margin-bottom: 10px;
   width: 225px;
   font-family: Georgia, "Times New Roman", Times, serif;
   list-style: none;
   text-transform: capitalize;
 }
 
-#tocView li:active,
-#tocView li.currentChapter
-{
+.md-content > div ul li {
+  padding: 5px 0;
+}
+
+#settingsPanel li {
+  font-size: 1em;
+  color: #f1f1f1;
+}
+
+#searchResults li {
+  margin-bottom: 10px;
+  width: 225px;
+  font-family: Georgia, "Times New Roman", Times, serif;
   list-style: none;
 }
 
-.list_item a {
-  color: #aaa;
-  text-decoration: none;
+#notes li {
+  color: #eee;
+  font-size: 12px;
+  width: 240px;
+  border-top: 1px #fff solid;
+  padding-top: 6px;
+  margin-bottom: 6px;
+}
+
+#tocView li:active,
+#tocView li.currentChapter {
+  list-style: none;
 }
 
 .list_item a.chapter {
@@ -322,27 +355,11 @@ input:-ms-placeholder { color: #454545; }
   font-size: 0.8em;
 }
 
-.list_item.currentChapter > a,
-.list_item a:hover {
-  color: #f1f1f1
-}
-
 /* #tocView li.openChapter > a, */
 .list_item a:hover {
   color: #e2e2e2;
 }
 
-.list_item ul {
-  padding-left:10px;
-  margin-top: 8px;
-  display: none;
-}
-
-.list_item.currentChapter > ul,
-.list_item.openChapter > ul {
-  display: block;
-}
-
 #tocView.hidden {
   display: none;
 }
@@ -357,14 +374,14 @@ input:-ms-placeholder { color: #454545; }
   user-select: none;
 }
 
-.toc_toggle:before {
+.toc_toggle::before {
   content: '▸';
   color: #fff;
   margin-right: -4px;
 }
 
-.currentChapter > .toc_toggle:before,
-.openChapter > .toc_toggle:before {
+.currentChapter > .toc_toggle::before,
+.openChapter > .toc_toggle::before {
   content: '▾';
 }
 
@@ -382,18 +399,6 @@ input:-ms-placeholder { color: #454545; }
   display: block;
 }
 
-#searchResults li {
-  margin-bottom:10px;
-  width: 225px;
-  font-family: Georgia, "Times New Roman", Times, serif;
-  list-style: none;
-}
-
-#searchResults a {
-  color: #aaa;
-  text-decoration: none;
-}
-
 #searchResults p {
   text-decoration: none;
   font-size: 12px;
@@ -405,10 +410,21 @@ input:-ms-placeholder { color: #454545; }
   color: #000;
 }
 
+.md-content > div p {
+  margin: 0;
+  padding: 10px 0;
+}
+
 #searchResults li > p {
   color: #aaa;
 }
 
+#notes li a {
+  color: #fff;
+  display: inline-block;
+  margin-left: 6px;
+}
+
 #searchResults li a:hover {
   color: #e2e2e2;
 }
@@ -419,22 +435,7 @@ input:-ms-placeholder { color: #454545; }
 }
 
 #notes {
- padding: 0 0 0 34px;
-}
-
-#notes li {
-  color: #eee;
-  font-size: 12px;
-  width: 240px;
-  border-top: 1px #fff solid;
-  padding-top: 6px;
-  margin-bottom: 6px;
-}
-
-#notes li a {
-  color: #fff;
-  display: inline-block;
-  margin-left: 6px;
+  padding: 0 0 0 34px;
 }
 
 #notes li a:hover {
@@ -454,8 +455,9 @@ input:-ms-placeholder { color: #454545; }
   border-radius: 5px;
 }
 
-#note-text[disabled], #note-text[disabled="disabled"]{
-    opacity: 0.5;
+#note-text[disabled],
+#note-text[disabled="disabled"] {
+  opacity: 0.5;
 }
 
 #note-anchor {
@@ -467,6 +469,22 @@ input:-ms-placeholder { color: #454545; }
   display: none;
 }
 
+.md-content h3 {
+  margin: 0;
+  padding: 6px;
+  text-align: center;
+  font-size: 22px;
+  font-weight: 300;
+  opacity: 0.8;
+  background: rgba(0, 0, 0, 0.1);
+  border-radius: 3px 3px 0 0;
+}
+
+.md-content > div ul {
+  margin: 0;
+  padding: 0 0 30px 20px;
+}
+
 #settingsPanel h3 {
   color: #f1f1f1;
   font-family: Georgia, "Times New Roman", Times, serif;
@@ -478,32 +496,24 @@ input:-ms-placeholder { color: #454545; }
   list-style-type: none;
 }
 
-#settingsPanel li {
-  font-size: 1em;
-  color: #f1f1f1;
-}
+#settingsPanel .xsmall { font-size: x-small; }
+#settingsPanel .small { font-size: small; }
+#settingsPanel .medium { font-size: medium; }
+#settingsPanel .large { font-size: large; }
+#settingsPanel .xlarge { font-size: x-large; }
 
-#settingsPanel .xsmall {  font-size: x-small; }
-#settingsPanel .small {  font-size: small; }
-#settingsPanel .medium {  font-size: medium; }
-#settingsPanel .large {  font-size: large; }
-#settingsPanel .xlarge {  font-size: x-large; }
-
-.highlight { background-color: yellow }
+.highlight { background-color: yellow; }
 
 .modal {
   position: fixed;
   top: 50%;
   left: 50%;
-  width: 50%;
   width: 630px;
-
   height: auto;
   z-index: 2000;
   visibility: hidden;
   margin-left: -320px;
   margin-top: -160px;
-
 }
 
 .overlay {
@@ -518,17 +528,16 @@ input:-ms-placeholder { color: #454545; }
   background: rgba(255, 255, 255, 0.8);
   -webkit-transition: all 0.3s;
   -moz-transition: all 0.3s;
-  -ms-transition: all 0.3s;
   transition: all 0.3s;
 }
 
 .md-show {
-    visibility: visible;
+  visibility: visible;
 }
 
 .md-show ~ .overlay {
-    opacity: 1;
-    visibility: visible;
+  opacity: 1;
+  visibility: visible;
 }
 
 /* Content styles */
@@ -541,17 +550,6 @@ input:-ms-placeholder { color: #454545; }
   height: 320px;
 }
 
-.md-content h3 {
-  margin: 0;
-  padding: 6px;
-  text-align: center;
-  font-size: 22px;
-  font-weight: 300;
-  opacity: 0.8;
-  background: rgba(0, 0, 0, 0.1);
-  border-radius: 3px 3px 0 0;
-}
-
 .md-content > div {
   padding: 15px 40px 30px;
   margin: 0;
@@ -559,20 +557,6 @@ input:-ms-placeholder { color: #454545; }
   font-size: 14px;
 }
 
-.md-content > div p {
-  margin: 0;
-  padding: 10px 0;
-}
-
-.md-content > div ul {
-  margin: 0;
-  padding: 0 0 30px 20px;
-}
-
-.md-content > div ul li {
-  padding: 5px 0;
-}
-
 .md-content button {
   display: block;
   margin: 0 auto;
@@ -588,7 +572,6 @@ input:-ms-placeholder { color: #454545; }
   opacity: 0;
   -webkit-transition: all 0.3s;
   -moz-transition: all 0.3s;
-  -ms-transition: all 0.3s;
   transition: all 0.3s;
 }
 
@@ -601,7 +584,6 @@ input:-ms-placeholder { color: #454545; }
 }
 
 .md-content > .closer {
-  font-size: 18px;
   position: absolute;
   right: 0;
   top: 0;
@@ -610,7 +592,7 @@ input:-ms-placeholder { color: #454545; }
 }
 
 @media only screen and (max-width: 1040px) and (orientation: portrait) {
-  #viewer{
+  #viewer {
     width: 80%;
     margin-left: 10%;
   }
@@ -622,7 +604,7 @@ input:-ms-placeholder { color: #454545; }
 }
 
 @media only screen and (max-width: 900px) {
-  #viewer{
+  #viewer {
     width: 60%;
     margin-left: 20%;
   }
@@ -637,7 +619,7 @@ input:-ms-placeholder { color: #454545; }
 }
 
 @media only screen and (max-width: 550px) {
-  #viewer{
+  #viewer {
     width: 80%;
     margin-left: 10%;
   }
@@ -661,9 +643,9 @@ input:-ms-placeholder { color: #454545; }
     -webkit-transform: translate(0, 0);
     -moz-transform: translate(0, 0);
     -ms-transform: translate(0, 0);
-    -webkit-transition: -webkit-transform .3s;
-    -moz-transition: -moz-transform .3s;
-    -ms-transition: -moz-transform .3s;
+    -webkit-transition: -webkit-transform 0.3s;
+    -moz-transition: -moz-transform 0.3s;
+    transition: -moz-transform 0.3s;
   }
 
   #main.closed {
@@ -672,11 +654,6 @@ input:-ms-placeholder { color: #454545; }
     -ms-transform: translate(260px, 0);
   }
 
-  #titlebar {
-    /* font-size: 16px; */
-    /* margin: 0 50px 0 50px; */
-  }
-
   #metainfo {
     font-size: 10px;
   }
@@ -689,130 +666,129 @@ input:-ms-placeholder { color: #454545; }
     font-size: 12px;
   }
 
-  #tocView > ul{
+  #tocView > ul {
     padding-left: 10px;
   }
 }
 
-
 /* For iPad portrait layouts only */
 @media only screen and (min-device-width: 481px) and (max-device-width: 1024px) and (orientation: portrait) {
-    #viewer iframe {
-        width: 460px;
-        height: 740px;
-    }
+  #viewer iframe {
+    width: 460px;
+    height: 740px;
+  }
 }
- /*For iPad landscape layouts only *//*
-@media only screen and (min-device-width: 481px) and (max-device-width: 1024px) and (orientation: landscape) {
-    #viewer iframe {
-        width: 460px;
-        height: 415px;
-    }
-}*/
 
 @media only screen
-and (min-device-width : 768px)
-and (max-device-width : 1024px)
-and (orientation : landscape)
-/*and (-webkit-min-device-pixel-ratio: 2)*/ {
-  #viewer{
+  and (min-device-width: 768px)
+  and (max-device-width: 1024px)
+  and (orientation: landscape)
+  /* and (-webkit-min-device-pixel-ratio: 2)*/ {
+  #viewer {
     width: 80%;
     margin-left: 10%;
   }
+
   #divider,
   #divider.show {
     display: none;
   }
 }
 
- /*For iPad landscape layouts only */
+/* For iPad landscape layouts only */
 @media only screen and (min-device-width: 481px) and (max-device-width: 1024px) and (orientation: landscape) {
-    #viewer iframe {
-        width: 960px;
-        height: 515px;
-    }
+  #viewer iframe {
+    width: 960px;
+    height: 515px;
+  }
 }
 
 /* For iPhone 6\6s portrait layouts only */
-@media only screen and (min-device-width : 375px) and (max-device-width : 667px) and (orientation: portrait) {
-    #viewer {
-      width: 300px;
-      height: 480px;
-    }
-    #viewer iframe {
-       width: 300px;
-       height: 480px;
-    }
+@media only screen and (min-device-width: 375px) and (max-device-width: 667px) and (orientation: portrait) {
+  #viewer {
+    width: 300px;
+    height: 480px;
+  }
+
+  #viewer iframe {
+   width: 300px;
+   height: 480px;
+  }
 }
 
 /* For iPhone 6\6s landscape layouts only */
-@media only screen and (min-device-width : 375px) and (max-device-width : 667px) and (orientation: landscape) {
-    #viewer {
-      width: 450px;
-      height: 300px;
-    }
-    #viewer iframe {
-       width: 450px;
-       height: 300px;
-    }
+@media only screen and (min-device-width: 375px) and (max-device-width: 667px) and (orientation: landscape) {
+  #viewer {
+    width: 450px;
+    height: 300px;
+  }
+
+  #viewer iframe {
+    width: 450px;
+    height: 300px;
+  }
 }
 
 /* For iPhone portrait layouts only */
 @media only screen and (max-device-width: 374px) and (orientation: portrait) {
-    #viewer {
-      width: 256px;
-      height: 432px;
-    }
-    #viewer iframe {
-       width: 256px;
-       height: 432px;
-    }
+  #viewer {
+    width: 256px;
+    height: 432px;
+  }
+
+  #viewer iframe {
+    width: 256px;
+    height: 432px;
+  }
 }
 
 /* For iPhone landscape layouts only */
 @media only screen and (max-device-width: 374px) and (orientation: landscape) {
-    #viewer iframe {
-        width: 256px;
-        height: 124px;
-    }
+  #viewer iframe {
+    width: 256px;
+    height: 124px;
+  }
 }
 
-[class^="icon-"]:before, [class*=" icon-"]:before {
-    font-family: "fontello";
-    font-style: normal;
-    font-weight: normal;
-    speak: none;
-    display: inline-block;
-    text-decoration: inherit;
-    width: 1em;
-    margin-right: 0.2em;
-    text-align: center;
-    /* For safety - reset parent styles, that can break glyph codes*/
-    font-variant: normal;
-    text-transform: none;
-    /* you can be more comfortable with increased icons size */
-    font-size: 112%;
+[class^="icon-"]::before,
+[class*=" icon-"]::before {
+  font-family: "fontello", serif;
+  font-style: normal;
+  font-weight: normal;
+  speak: none;
+  display: inline-block;
+  text-decoration: inherit;
+  width: 1em;
+  margin-right: 0.2em;
+  text-align: center;
+
+  /* For safety - reset parent styles, that can break glyph codes */
+  font-variant: normal;
+  text-transform: none;
+
+  /* you can be more comfortable with increased icons size */
+  font-size: 112%;
 }
 
-.icon-search:before { content: '\e807'; } /* '' */
-.icon-resize-full-1:before { content: '\e804'; } /* '' */
-.icon-cancel-circled2:before { content: '\e80f'; } /* '' */
-.icon-link:before { content: '\e80d'; } /* '' */
-.icon-bookmark:before { content: '\e805'; } /* '' */
-.icon-bookmark-empty:before { content: '\e806'; } /* '' */
-.icon-download-cloud:before { content: '\e811'; } /* '' */
-.icon-edit:before { content: '\e814'; } /* '' */
-.icon-menu:before { content: '\e802'; } /* '' */
-.icon-cog:before { content: '\e813'; } /* '' */
-.icon-resize-full:before { content: '\e812'; } /* '' */
-.icon-cancel-circled:before { content: '\e80e'; } /* '' */
-.icon-up-dir:before { content: '\e80c'; } /* '' */
-.icon-right-dir:before { content: '\e80b'; } /* '' */
-.icon-angle-right:before { content: '\e809'; } /* '' */
-.icon-angle-down:before { content: '\e80a'; } /* '' */
-.icon-right:before { content: '\e815'; } /* '' */
-.icon-list-1:before { content: '\e803'; } /* '' */
-.icon-list-numbered:before { content: '\e801'; } /* '' */
-.icon-columns:before { content: '\e810'; } /* '' */
-.icon-list:before { content: '\e800'; } /* '' */
-.icon-resize-small:before { content: '\e808'; } /* '' */
+.icon-search::before { content: '\e807'; } /* '' */
+.icon-resize-full-1::before { content: '\e804'; } /* '' */
+.icon-cancel-circled2::before { content: '\e80f'; } /* '' */
+.icon-link::before { content: '\e80d'; } /* '' */
+.icon-bookmark::before { content: '\e805'; } /* '' */
+.icon-bookmark-empty::before { content: '\e806'; } /* '' */
+.icon-download-cloud::before { content: '\e811'; } /* '' */
+.icon-edit::before { content: '\e814'; } /* '' */
+.icon-menu::before { content: '\e802'; } /* '' */
+.icon-cog::before { content: '\e813'; } /* '' */
+.icon-resize-full::before { content: '\e812'; } /* '' */
+.icon-cancel-circled::before { content: '\e80e'; } /* '' */
+.icon-up-dir::before { content: '\e80c'; } /* '' */
+.icon-right-dir::before { content: '\e80b'; } /* '' */
+.icon-angle-right::before { content: '\e809'; } /* '' */
+.icon-angle-down::before { content: '\e80a'; } /* '' */
+.icon-right::before { content: '\e815'; } /* '' */
+.icon-list-1::before { content: '\e803'; } /* '' */
+.icon-list-numbered::before { content: '\e801'; } /* '' */
+.icon-columns::before { content: '\e810'; } /* '' */
+.icon-list::before { content: '\e800'; } /* '' */
+.icon-resize-small::before { content: '\e808'; } /* '' */

cps/static/css/style.css

@@ -1,7 +1,7 @@
 
 .tooltip.bottom .tooltip-inner {
   font-size: 13px;
-  font-family: Open Sans Semibold,Helvetica Neue,Helvetica,Arial,sans-serif;
+  font-family: Open Sans Semibold, Helvetica Neue, Helvetica, Arial, sans-serif;
   -webkit-font-smoothing: antialiased;
   -moz-osx-font-smoothing: grayscale;
   padding: 3px 10px;
@@ -28,6 +28,11 @@ html.http-error {
   height: 100%;
 }
 
+body {
+  background: #f2f2f2;
+  margin-bottom: 40px;
+}
+
 .http-error body {
   margin: 0;
   height: 100%;
@@ -41,17 +46,39 @@ html.http-error {
   text-align: center;
 }
 
-body {
-  background: #f2f2f2;
-  margin-bottom: 40px;
-}
-
 body h2 {
   font-weight: normal;
-  color:#444;
+  color: #444;
 }
 
-a { color: #45b29d; }
+a,
+.danger,
+.book-remove,
+.editable-empty,
+.editable-empty:hover { color: #45b29d; }
+.user-remove:hover { color: #23527c; }
+.btn-default a { color: #444; }
+.panel-title > a { text-decoration: none; }
+
+.navigation li a {
+  color: #444;
+  text-decoration: none;
+  display: block;
+  padding: 10px;
+}
+
+.btn-default a:hover {
+  color: #45b29d;
+  text-decoration: None;
+}
+
+.btn-default:hover {
+  color: #45b29d;
+}
+
+.editable-click,
+a.editable-click,
+a.editable-click:hover { border-bottom: None; }
 
 .navigation .nav-head {
   text-transform: uppercase;
@@ -63,11 +90,18 @@ a { color: #45b29d; }
   border-top: 1px solid #ccc;
   padding-top: 20px;
 }
-.navigation li a {
-  color: #444;
+
+.book-meta .tags a { display: inline; }
+table .bg-primary a { color: #fff; }
+table .bg-dark-danger a { color: #fff; }
+.book-meta .identifiers a { display: inline; }
+
+.navigation .create-shelf a {
+  color: #fff;
+  background: #45b29d;
+  padding: 10px 20px;
+  border-radius: 5px;
   text-decoration: none;
-  display: block;
-  padding: 10px;
 }
 
 .navigation li a:hover {
@@ -83,42 +117,68 @@ a { color: #45b29d; }
   text-align: center;
 }
 
-.navigation .create-shelf a {
-  color: #fff;
-  background: #45b29d;
-  padding: 10px 20px;
-  border-radius: 5px;
-  text-decoration: none;
+.row.display-flex {
+  display: flex;
+  flex-wrap: wrap;
 }
 
 .container-fluid img {
   display: block;
   max-width: 100%;
   height: auto;
+  max-height: 100%;
 }
 
-.container-fluid .discover{ margin-bottom: 50px; }
+.container-fluid .discover { margin-bottom: 50px; }
 .container-fluid .new-books { border-top: 1px solid #ccc; }
 .container-fluid .new-books h2 { margin: 50px 0 0 0; }
-.container-fluid .book { margin-top: 20px; }
+
+.container-fluid .book {
+  margin-top: 20px;
+  display: flex;
+  flex-direction: column;
+}
+.cover { margin-bottom: 10px; }
 
 .container-fluid .book .cover {
   height: 225px;
   position: relative;
 }
 
-.container-fluid .book .cover img {
+.author-link img {
+  display: block;
+  height: 100%;
+}
+
+.container-fluid .book .cover span.img {
+  bottom: 0;
+  height: 100%;
+  position: absolute;
+}
+.author-bio img { margin: 0 1em 1em 0; }
+
+.container-fluid .single .cover img {
   border: 1px solid #fff;
   box-sizing: border-box;
+  -webkit-box-shadow: 0 5px 8px -6px #777;
+  -moz-box-shadow: 0 5px 8px -6px #777;
+  box-shadow: 0 5px 8px -6px #777;
+}
+
+.container-fluid .book .cover span img {
+  position: relative;
+  top: 0;
+  left: 0;
   height: 100%;
-  bottom: 0;
-  position: absolute;
+  border: 1px solid #fff;
+  box-sizing: border-box;
   -webkit-box-shadow: 0 5px 8px -6px #777;
   -moz-box-shadow: 0 5px 8px -6px #777;
   box-shadow: 0 5px 8px -6px #777;
 }
 
 .container-fluid .book .meta { margin-top: 10px; }
+.media-body p { text-align: justify; }
 .container-fluid .book .meta p { margin: 0; }
 
 .container-fluid .book .meta .title {
@@ -127,6 +187,12 @@ a { color: #45b29d; }
   color: #444;
 }
 
+.container-fluid .book .meta .series {
+  font-weight: 400;
+  font-size: 12px;
+  color: #444;
+}
+
 .container-fluid .book .meta .author {
   font-size: 12px;
   color: #999;
@@ -135,9 +201,10 @@ a { color: #45b29d; }
 .container-fluid .book .meta .rating { margin-top: 5px; }
 .rating .glyphicon-star-empty { color: #444; }
 .rating .glyphicon-star.good { color: #444; }
-.rating-clear .glyphicon-remove { color: #333  }
+.rating-clear .glyphicon-remove { color: #333; }
 
-.container-fluid .author .author-hidden, .container-fluid .author .author-hidden-divider { display: none; }
+.container-fluid .author .author-hidden,
+.container-fluid .author .author-hidden-divider { display: none; }
 
 .navbar-brand {
   font-family: 'Grand Hotel', cursive;
@@ -151,7 +218,7 @@ a { color: #45b29d; }
   border-top: 1px solid #ccc;
 }
 
-.more-stuff>li { margin-bottom: 10px; }
+.more-stuff > li { margin-bottom: 10px; }
 .navbar-collapse.in .navbar-nav { margin: 0; }
 
 span.glyphicon.glyphicon-tags {
@@ -161,34 +228,38 @@ span.glyphicon.glyphicon-tags {
 }
 
 .book-meta { padding-bottom: 20px; }
-.book-meta .tags a { display: inline; }
-.book-meta .identifiers a { display: inline; }
 
-.container-fluid .single .cover img {
-  border: 1px solid #fff;
-  box-sizing: border-box;
-  -webkit-box-shadow: 0 5px 8px -6px #777;
-  -moz-box-shadow: 0 5px 8px -6px #777;
-  box-shadow: 0 5px 8px -6px #777;
+.navbar-default .navbar-toggle .icon-bar { background-color: #000; }
+.navbar-default .navbar-toggle { border-color: #000; }
+
+.cover .badge {
+  position: absolute;
+  top: 2px;
+  left: 2px;
+  color: #000;
+  border-radius: 10px;
+  background-color: #fff;
 }
 
-.navbar-default .navbar-toggle .icon-bar {background-color: #000; }
-.navbar-default .navbar-toggle {border-color: #000; }
-.cover { margin-bottom: 10px; }
-.cover .badge{
-   position: absolute;
-   top: 2px;
-   left: 2px;
-   background-color: #777;
+.cover .read {
+  left: auto;
+  right: 2px;
+  width: 17px;
+  height: 17px;
+  display: inline-block;
+  padding: 2px;
 }
-.cover-height { max-height: 100px;}
+.cover-height { max-height: 100px; }
 
 .col-sm-2 a .cover-small {
   margin: 5px;
   max-height: 200px;
 }
 
-.btn-file {position: relative; overflow: hidden;}
+.btn-file {
+  position: relative;
+  overflow: hidden;
+}
 
 .btn-file input[type=file] {
   position: absolute;
@@ -206,24 +277,60 @@ span.glyphicon.glyphicon-tags {
   display: block;
 }
 
-.btn-toolbar .btn,.discover .btn { margin-bottom: 5px; }
-.button-link {color: #fff; }
-.btn-primary:hover, .btn-primary:focus, .btn-primary:active, .btn-primary.active, .open .dropdown-toggle.btn-primary{ background-color: #1C5484; }
-.btn-primary.disabled, .btn-primary[disabled], fieldset[disabled] .btn-primary, .btn-primary.disabled:hover, .btn-primary[disabled]:hover, fieldset[disabled] .btn-primary:hover, .btn-primary.disabled:focus, .btn-primary[disabled]:focus, fieldset[disabled] .btn-primary:focus, .btn-primary.disabled:active, .btn-primary[disabled]:active, fieldset[disabled] .btn-primary:active, .btn-primary.disabled.active, .btn-primary[disabled].active, fieldset[disabled] .btn-primary.active { background-color: #89B9E2; }
-.btn-toolbar>.btn+.btn, .btn-toolbar>.btn-group+.btn, .btn-toolbar>.btn+.btn-group, .btn-toolbar>.btn-group+.btn-group { margin-left: 0px; }
-.panel-body {background-color: #f5f5f5; }
-.spinner {margin: 0 41%; }
-.spinner2 {margin: 0 41%; }
-.intend-form { margin-left:20px; }
-table .bg-dark-danger {background-color: #d9534f; color: #fff; }
-table .bg-dark-danger a {color: #fff; }
-table .bg-dark-danger:hover {background-color: #c9302c; }
-table .bg-primary:hover {background-color: #1C5484; }
-table .bg-primary a {color: #fff; }
-.block-label {display: block;}
-.fake-input {position: absolute; pointer-events: none; top: 0; }
+.btn-toolbar .btn,
+.discover .btn { margin-bottom: 5px; }
+.button-link { color: #fff; }
 
-input.pill { position: absolute; opacity: 0; }
+.btn-primary:hover,
+.btn-primary:focus,
+.btn-primary:active,
+.btn-primary.active,
+.open .dropdown-toggle.btn-primary { background-color: #1c5484; }
+
+.btn-primary.disabled,
+.btn-primary[disabled],
+fieldset[disabled] .btn-primary,
+.btn-primary.disabled:hover,
+.btn-primary[disabled]:hover,
+fieldset[disabled] .btn-primary:hover,
+.btn-primary.disabled:focus,
+.btn-primary[disabled]:focus,
+fieldset[disabled] .btn-primary:focus,
+.btn-primary.disabled:active,
+.btn-primary[disabled]:active,
+fieldset[disabled] .btn-primary:active,
+.btn-primary.disabled.active,
+.btn-primary[disabled].active,
+fieldset[disabled] .btn-primary.active { background-color: #89b9e2; }
+
+.btn-toolbar > .btn + .btn,
+.btn-toolbar > .btn-group + .btn,
+.btn-toolbar > .btn + .btn-group,
+.btn-toolbar > .btn-group + .btn-group { margin-left: 0; }
+
+.panel-body { background-color: #f5f5f5; }
+.spinner { margin: 0 41%; }
+.spinner2 { margin: 0 41%; }
+.intend-form { margin-left: 20px; }
+
+table .bg-dark-danger {
+  background-color: #d9534f;
+  color: #fff;
+}
+table .bg-dark-danger:hover { background-color: #c9302c; }
+table .bg-primary:hover { background-color: #1c5484; }
+.block-label { display: block; }
+
+.fake-input {
+  position: absolute;
+  pointer-events: none;
+  top: 0;
+}
+
+input.pill {
+  position: absolute;
+  opacity: 0;
+}
 
 input.pill + label {
   border: 2px solid #45b29d;
@@ -245,12 +352,18 @@ input.pill:checked + label {
 
 input.pill:not(:checked) + label .glyphicon { display: none; }
 
-.author-bio img { margin: 0 1em 1em 0; }
-.author-link { display: inline-block; margin-top: 10px; width: 100px; }
-.author-link img { display: block; height: 100%; }
-#remove-from-shelves .btn, #shelf-action-errors { margin-left: 5px; }
+.author-link {
+  display: inline-block;
+  margin-top: 10px;
+  width: 100px;
+}
 
-.tags_click, .serie_click, .language_click { margin-right: 5px; }
+#remove-from-shelves .btn,
+#shelf-action-errors { margin-left: 5px; }
+
+.tags_click,
+.serie_click,
+.language_click { margin-right: 5px; }
 
 #meta-info {
   height: 600px;
@@ -258,7 +371,6 @@ input.pill:not(:checked) + label .glyphicon { display: none; }
 }
 
 .media-list { padding-right: 15px; }
-.media-body p { text-align: justify; }
 
 #meta-info img {
   max-height: 150px;
@@ -271,22 +383,23 @@ input.pill:not(:checked) + label .glyphicon { display: none; }
 #btn-upload-format { display: none; }
 .upload-cover-input-text { display: initial; }
 #btn-upload-cover { display: none; }
-.panel-title > a { text-decoration: none; }
+
 .editable-buttons {
-  display:inline-block;
+  display: inline-block;
   margin-left: 7px;
 }
 
-.editable-input { display:inline-block; }
+.editable-input { display: inline-block; }
 
 .editable-cancel {
-  margin-bottom: 0px !important;
+  margin-bottom: 0 !important;
   margin-left: 7px !important;
 }
 
-.editable-submit { margin-bottom: 0px !important; }
+.editable-submit { margin-bottom: 0 !important; }
 .filterheader { margin-bottom: 20px; }
 .errorlink { margin-top: 20px; }
+.emailconfig { margin-top: 10px; }
 
 .modal-body .comments {
   max-height: 300px;
@@ -294,7 +407,7 @@ input.pill:not(:checked) + label .glyphicon { display: none; }
 }
 
 div.log {
-  font-family: Courier New;
+  font-family: Courier New, serif;
   font-size: 12px;
   box-sizing: border-box;
   height: 700px;
@@ -303,4 +416,3 @@ div.log {
   white-space: nowrap;
   padding: 0.5em;
 }
-

cps/static/js/caliBlur.js

@@ -145,44 +145,16 @@ if ($("body.book").length > 0) {
     $(".blur-wrapper")
         .prepend('<div><img alt="Blurred cover" class="bg-blur" src="' + cover + '"></div>');
 
-    // Fix-up book detail headings
-    publisher = $(".publishers p span").text().split(":");
-    $(".publishers p span").remove();
-    $.each(publisher, function (i, val) {
-        $(".publishers").append("<span>" + publisher[i] + "</span>");
-    });
-    $(".publishers span:nth-child(3)").text(function () {
-        return $(this).text().replace(/^\s+|^\t+|\t+|\s+$/g, "");
-    });
-
-    // Fix-up book custom colums headings
-    // real_custom_column = $( '.real_custom_columns' ).text().split( ':' );
-    real_custom_column = $(".real_custom_columns");
-    // $( ".real_custom_columns" ).remove();
-    $.each(real_custom_column, function (i, val) {
-        var split = $(this).text().split(":");
-        real_cc_key = split.shift();
-        real_cc_value = split.join(":");
-        $(this).text("");
-        if (real_cc_value != "") {
-            $(this).append("<span>" + real_cc_key + "</span><span>" + real_cc_value + "</span>");
+    // Metadata Fields - Publishers, Published, Languages and Custom
+    $('.publishers, .publishing-date, .real_custom_columns, .languages').each(function () {
+        var splitText = $(this).text().split(':');
+        var label = splitText.shift().trim();
+        var value = splitText.join(':').trim();
+        // Preserve Links
+        if ($(this).find('a').length) {
+            value = $(this).find('a').first().removeClass();
         }
-    });
-    //$( '.real_custom_columns:nth-child(3)' ).text(function() {
-    //return $(this).text().replace(/^\s+|^\t+|\t+|\s+$/g, "");
-    //});
-
-    published = $(".publishing-date p")
-        .text().split(": ");
-    $(".publishing-date p").remove();
-    $.each(published, function (i, val) {
-        $(".publishing-date").append("<span>" + published[i] + "</span>");
-    });
-
-    languages = $(".languages p span").text().split(": ");
-    $(".languages p span").remove();
-    $.each(languages, function (i, val) {
-        $(".languages").append("<span>" + languages[i] + "</span>");
+        $(this).html('<span>' + label + '</span><span></span>').find('span').last().append(value);
     });
 
     $(".book-meta h2:first").clone()
@@ -246,11 +218,6 @@ if ($("body.book").length > 0) {
         $("#add-to-shelves").toggle();
     });
 
-    // Fix formatting error on book detail languages
-    if (!$(".book-meta > .bookinfo > .languages > span:last-of-type").text().startsWith(" ")) {
-        $(".book-meta > .bookinfo > .languages > span:last-of-type").prepend(" ");
-    }
-
     //Work to reposition dropdowns. Does not currently solve for
     //screen resizing
     function dropdownToggle() {
@@ -710,7 +677,7 @@ $(".navbar-collapse.collapse.in").before('<div class="sidebar-backdrop"></div>')
 // Get rid of leading white space
 recentlyAdded = $("#nav_new a:contains('Recently')").text().trim();
 $("#nav_new a:contains('Recently')").contents().filter(function () {
-    return this.nodeType == 3
+    return this.nodeType === 3
 }).each(function () {
     this.textContent = this.textContent.replace(" Recently Added", recentlyAdded);
 });

cps/static/js/edit_books.js

@@ -1,7 +1,7 @@
 /**
  * Created by SpeedProg on 05.04.2015.
  */
-/* global Bloodhound, language, Modernizr, tinymce */
+/* global Bloodhound, language, Modernizr, tinymce, getPath */
 
 if ($("#description").length) {
     tinymce.init({
@@ -78,10 +78,10 @@ function prefixedSource(prefix, query, cb, bhAdapter) {
     });
 }
 
-function getPath() {
+/*function getPath() {
     var jsFileLocation = $("script[src*=edit_books]").attr("src");  // the js file path
     return jsFileLocation.substr(0, jsFileLocation.search("/static/js/edit_books.js"));   // the js folder path
-}
+}*/
 
 var authors = new Bloodhound({
     name: "authors",
@@ -249,18 +249,26 @@ promisePublishers.done(function() {
     );
 });
 
-$("#search").on("change input.typeahead:selected", function() {
+$("#search").on("change input.typeahead:selected", function(event) {
+    if (event.target.type === "search" && event.target.tagName === "INPUT") {
+        return;
+    }
     var form = $("form").serialize();
     $.getJSON( getPath() + "/get_matching_tags", form, function( data ) {
         $(".tags_click").each(function() {
-            if ($.inArray(parseInt($(this).children("input").first().val(), 10), data.tags) === -1 ) {
-                if (!($(this).hasClass("active"))) {
-                    $(this).addClass("disabled");
+            if ($.inArray(parseInt($(this).val(), 10), data.tags) === -1) {
+                if (!$(this).prop("selected")) {
+                    $(this).prop("disabled", true);
                 }
             } else {
-                $(this).removeClass("disabled");
+                $(this).prop("disabled", false);
             }
         });
+        $("#include_tag option:selected").each(function () {
+            $("#exclude_tag").find("[value=" + $(this).val() + "]").prop("disabled", true);
+        });
+        $("#include_tag").selectpicker("refresh");
+        $("#exclude_tag").selectpicker("refresh");
     });
 });
 

cps/static/js/filter_grid.js

@@ -24,31 +24,47 @@ var $list = $("#list").isotope({
 });
 
 $("#desc").click(function() {
+    var page = $(this).data("id");
+    $.ajax({
+        method:"post",
+        contentType: "application/json; charset=utf-8",
+        dataType: "json",
+        url: window.location.pathname + "/../../ajax/view",
+        data: "{\"" + page + "\": {\"dir\": \"desc\"}}",
+    });
     $list.isotope({
         sortBy: "name",
         sortAscending: true
     });
-    return;
 });
 
 $("#asc").click(function() {
+    var page = $(this).data("id");
+    $.ajax({
+        method:"post",
+        contentType: "application/json; charset=utf-8",
+        dataType: "json",
+        url: window.location.pathname + "/../../ajax/view",
+        data: "{\"" + page + "\": {\"dir\": \"asc\"}}",
+    });
     $list.isotope({
         sortBy: "name",
         sortAscending: false
     });
-    return;
 });
 
 $("#all").click(function() {
     // go through all elements and make them visible
     $list.isotope({ filter: function() {
         return true;
-      } })
+    }
+    });
 });
 
 $(".char").click(function() {
     var character = this.innerText;
     $list.isotope({ filter: function() {
-        return this.attributes["data-id"].value.charAt(0).toUpperCase() == character;
-      } })
+        return this.attributes["data-id"].value.charAt(0).toUpperCase() === character;
+    }
+    });
 });

cps/static/js/filter_list.js

@@ -19,6 +19,10 @@ var direction = 0;  // Descending order
 var sort = 0;       // Show sorted entries
 
 $("#sort_name").click(function() {
+    var className = $("h1").attr("Class") + "_sort_name";
+    var obj = {};
+    obj[className] = sort;
+
     var count = 0;
     var index = 0;
     var store;
@@ -40,9 +44,7 @@ $("#sort_name").click(function() {
             count++;
         }
     });
-    /*listItems.sort(function(a,b){
-        return $(a).children()[1].innerText.localeCompare($(b).children()[1].innerText)
-    });*/
+
     // Find count of middle element
     if (count > 20) {
         var middle = parseInt(count / 2, 10) + (count % 2);
@@ -66,6 +68,14 @@ $("#desc").click(function() {
     if (direction === 0) {
         return;
     }
+    var page = $(this).data("id");
+    $.ajax({
+        method:"post",
+        contentType: "application/json; charset=utf-8",
+        dataType: "json",
+        url: window.location.pathname + "/../../ajax/view",
+        data: "{\"" + page + "\": {\"dir\": \"desc\"}}",
+    });
     var index = 0;
     var list = $("#list");
     var second = $("#second");
@@ -78,7 +88,7 @@ $("#desc").click(function() {
     // Find count of middle element
     var count = $(".row:visible").length;
     if (count > 20) {
-        middle = parseInt(count / 2) + (count % 2);
+        middle = parseInt(count / 2, 10) + (count % 2);
 
         //var middle = parseInt(count / 2) + (count % 2);
         // search for the middle of all visible elements
@@ -102,9 +112,18 @@ $("#desc").click(function() {
 
 
 $("#asc").click(function() {
+
     if (direction === 1) {
         return;
     }
+    var page = $(this).data("id");
+    $.ajax({
+        method:"post",
+        contentType: "application/json; charset=utf-8",
+        dataType: "json",
+        url: window.location.pathname + "/../../ajax/view",
+        data: "{\"" + page + "\": {\"dir\": \"asc\"}}",
+    });
     var index = 0;
     var list = $("#list");
     var second = $("#second");
@@ -116,7 +135,7 @@ $("#asc").click(function() {
     // Find count of middle element
     var count = $(".row:visible").length;
     if (count > 20) {
-        var middle = parseInt(count / 2) + (count % 2);
+        var middle = parseInt(count / 2, 10) + (count % 2);
 
         //var middle = parseInt(count / 2) + (count % 2);
         // search for the middle of all visible elements
@@ -131,7 +150,6 @@ $("#asc").click(function() {
         });
 
         // middle = parseInt(elementLength / 2) + (elementLength % 2);
-
         list.append(reversed.slice(0, index));
         second.append(reversed.slice(index, elementLength));
     } else {

cps/static/js/get_meta.js

@@ -108,7 +108,7 @@ $(function () {
                         tags: result.volumeInfo.categories || [],
                         rating: result.volumeInfo.averageRating || 0,
                         cover: result.volumeInfo.imageLinks ?
-                            result.volumeInfo.imageLinks.thumbnail : "/static/generic_cover.jpg",
+                            result.volumeInfo.imageLinks.thumbnail : location + "/../../../static/generic_cover.jpg",
                         url: "https://books.google.com/books?id=" + result.id,
                         source: {
                             id: "google",
@@ -138,8 +138,8 @@ $(function () {
                         seriesTitle = result.series.title;
                     }
                     var dateFomers = result.pubdate.split("-");
-                    var publishedYear = parseInt(dateFomers[0]);
-                    var publishedMonth = parseInt(dateFomers[1]);
+                    var publishedYear = parseInt(dateFomers[0], 10);
+                    var publishedMonth = parseInt(dateFomers[1], 10);
                     var publishedDate = new Date(publishedYear, publishedMonth - 1, 1);
 
                     publishedDate = formatDate(publishedDate);
@@ -194,8 +194,8 @@ $(function () {
                     } else {
                         dateFomers = result.date_added.split("-");
                     }
-                    var publishedYear = parseInt(dateFomers[0]);
-                    var publishedMonth = parseInt(dateFomers[1]);
+                    var publishedYear = parseInt(dateFomers[0], 10);
+                    var publishedMonth = parseInt(dateFomers[1], 10);
                     var publishedDate = new Date(publishedYear, publishedMonth - 1, 1);
 
                     publishedDate = formatDate(publishedDate);
@@ -253,7 +253,7 @@ $(function () {
     }
 
     function dbSearchBook (title) {
-        var apikey = "0df993c66c0c636e29ecbb5344252a4a";
+        var apikey = "054022eaeae0b00e0fc068c0c0a2102a";
         $.ajax({
             url: douban + dbSearch + "?apikey=" + apikey + "&q=" + title + "&fields=all&count=10",
             type: "GET",

cps/static/js/kthoom.js

@@ -141,9 +141,27 @@ var createURLFromArray = function(array, mimeType) {
 kthoom.ImageFile = function(file) {
     this.filename = file.filename;
     var fileExtension = file.filename.split(".").pop().toLowerCase();
-    this.mimeType = fileExtension === "png" ? "image/png" :
-        (fileExtension === "jpg" || fileExtension === "jpeg") ? "image/jpeg" :
-            fileExtension === "gif" ? "image/gif" : fileExtension === "svg" ? "image/xml+svg" : undefined;
+    switch (fileExtension) {
+        case "jpg":
+        case "jpeg":
+            this.mimeType = "image/jpeg";
+            break;
+        case "png":
+            this.mimeType = "image/png";
+            break;
+        case "gif":
+            this.mimeType = "image/gif";
+            break;
+        case "svg":
+            this.mimeType = "image/svg+xml";
+            break;
+        case "webp":
+            this.mimeType = "image/webp";
+            break;
+        default:
+            this.mimeType = undefined;
+            break;
+    }
     if ( this.mimeType !== undefined) {
         this.dataURI = createURLFromArray(file.fileData, this.mimeType);
         this.data = file;
@@ -331,7 +349,7 @@ function setImage(url) {
                         $("#mainText").innerHTML("<iframe style=\"width:100%;height:700px;border:0\" src=\"data:text/html," + escape(xhr.responseText) + "\"></iframe>");
                     };
                     xhr.send(null);
-                } else if (!/(jpg|jpeg|png|gif)$/.test(imageFiles[currentImage].filename) && imageFiles[currentImage].data.uncompressedSize < 10 * 1024) {
+                } else if (!/(jpg|jpeg|png|gif|webp)$/.test(imageFiles[currentImage].filename) && imageFiles[currentImage].data.uncompressedSize < 10 * 1024) {
                     xhr.open("GET", url, true);
                     xhr.onload = function() {
                         $("#mainText").css("display", "");
@@ -635,6 +653,14 @@ function init(filename) {
     // Focus the scrollable area so that keyboard scrolling work as expected
     $("#mainContent").focus();
 
+    $("#mainContent").swipe( {
+        swipeRight:function() {
+            showLeftPage();
+        },
+        swipeLeft:function() {
+            showRightPage();
+        },
+    });
     $("#mainImage").click(function(evt) {
         // Firefox does not support offsetX/Y so we have to manually calculate
         // where the user clicked in the image.

cps/static/js/libs/bootstrap-datepicker/locales/bootstrap-datepicker.el.min.js

@@ -0,0 +1 @@
+!function(a){a.fn.datepicker.dates.el={days:["Κυριακή","Δευτέρα","Τρίτη","Τετάρτη","Πέμπτη","Παρασκευή","Σάββατο"],daysShort:["Κυρ","Δευ","Τρι","Τετ","Πεμ","Παρ","Σαβ"],daysMin:["Κυ","Δε","Τρ","Τε","Πε","Πα","Σα"],months:["Ιανουάριος","Φεβρουάριος","Μάρτιος","Απρίλιος","Μάιος","Ιούνιος","Ιούλιος","Αύγουστος","Σεπτέμβριος","Οκτώβριος","Νοέμβριος","Δεκέμβριος"],monthsShort:["Ιαν","Φεβ","Μαρ","Απρ","Μάι","Ιουν","Ιουλ","Αυγ","Σεπ","Οκτ","Νοε","Δεκ"],today:"Σήμερα",clear:"Καθαρισμός",weekStart:1,format:"d/m/yyyy"}}(jQuery);

cps/static/js/libs/bootstrap-datepicker/locales/bootstrap-datepicker.tr.min.js

@@ -0,0 +1 @@
+!function(a){a.fn.datepicker.dates.tr={days:["Pazar","Pazartesi","Salı","Çarşamba","Perşembe","Cuma","Cumartesi"],daysShort:["Pz","Pzt","Sal","Çrş","Prş","Cu","Cts"],daysMin:["Pz","Pzt","Sa","Çr","Pr","Cu","Ct"],months:["Ocak","Şubat","Mart","Nisan","Mayıs","Haziran","Temmuz","Ağustos","Eylül","Ekim","Kasım","Aralık"],monthsShort:["Oca","Şub","Mar","Nis","May","Haz","Tem","Ağu","Eyl","Eki","Kas","Ara"],today:"Bugün",clear:"Temizle",weekStart:1,format:"dd.mm.yyyy"}}(jQuery);

cps/static/js/libs/bootstrap-select/defaults-cs.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,n){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return n(e)}):"object"==typeof module&&module.exports?module.exports=n(require("jquery")):n(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Vyberte ze seznamu",noneResultsText:"Pro hled\xe1n\xed {0} nebyly nalezeny \u017e\xe1dn\xe9 v\xfdsledky",countSelectedText:"Vybran\xe9 {0} z {1}",maxOptionsText:["Limit p\u0159ekro\u010den ({n} {var} max)","Limit skupiny p\u0159ekro\u010den ({n} {var} max)",["polo\u017eek","polo\u017eka"]],multipleSeparator:", ",selectAllText:"Vybrat v\u0161e",deselectAllText:"Zru\u0161it v\xfdb\u011br"}});

cps/static/js/libs/bootstrap-select/defaults-de.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Bitte w\xe4hlen...",noneResultsText:"Keine Ergebnisse f\xfcr {0}",countSelectedText:function(e,t){return 1==e?"{0} Element ausgew\xe4hlt":"{0} Elemente ausgew\xe4hlt"},maxOptionsText:function(e,t){return[1==e?"Limit erreicht ({n} Element max.)":"Limit erreicht ({n} Elemente max.)",1==t?"Gruppen-Limit erreicht ({n} Element max.)":"Gruppen-Limit erreicht ({n} Elemente max.)"]},selectAllText:"Alles ausw\xe4hlen",deselectAllText:"Nichts ausw\xe4hlen",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-es.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,o){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return o(e)}):"object"==typeof module&&module.exports?module.exports=o(require("jquery")):o(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"No hay selecci\xf3n",noneResultsText:"No hay resultados {0}",countSelectedText:"Seleccionados {0} de {1}",maxOptionsText:["L\xedmite alcanzado ({n} {var} max)","L\xedmite del grupo alcanzado({n} {var} max)",["elementos","element"]],multipleSeparator:", ",selectAllText:"Seleccionar Todos",deselectAllText:"Desmarcar Todos"}});

cps/static/js/libs/bootstrap-select/defaults-fi.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Ei valintoja",noneResultsText:"Ei hakutuloksia {0}",countSelectedText:function(e,t){return 1==e?"{0} valittu":"{0} valitut"},maxOptionsText:function(e,t){return["Valintojen maksimim\xe4\xe4r\xe4 ({n} saavutettu)","Ryhm\xe4n maksimim\xe4\xe4r\xe4 ({n} saavutettu)"]},selectAllText:"Valitse kaikki",deselectAllText:"Poista kaikki",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-fr.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Aucune s\xe9lection",noneResultsText:"Aucun r\xe9sultat pour {0}",countSelectedText:function(e,t){return 1<e?"{0} \xe9l\xe9ments s\xe9lectionn\xe9s":"{0} \xe9l\xe9ment s\xe9lectionn\xe9"},maxOptionsText:function(e,t){return[1<e?"Limite atteinte ({n} \xe9l\xe9ments max)":"Limite atteinte ({n} \xe9l\xe9ment max)",1<t?"Limite du groupe atteinte ({n} \xe9l\xe9ments max)":"Limite du groupe atteinte ({n} \xe9l\xe9ment max)"]},multipleSeparator:", ",selectAllText:"Tout s\xe9lectionner",deselectAllText:"Tout d\xe9s\xe9lectionner"}});

cps/static/js/libs/bootstrap-select/defaults-hu.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"V\xe1lasszon!",noneResultsText:"Nincs tal\xe1lat {0}",countSelectedText:function(e,t){return"{0} elem kiv\xe1lasztva"},maxOptionsText:function(e,t){return["Legfeljebb {n} elem v\xe1laszthat\xf3","A csoportban legfeljebb {n} elem v\xe1laszthat\xf3"]},selectAllText:"Mind",deselectAllText:"Egyik sem",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-it.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Nessuna selezione",noneResultsText:"Nessun risultato per {0}",countSelectedText:function(e,t){return 1==e?"Selezionato {0} di {1}":"Selezionati {0} di {1}"},maxOptionsText:["Limite raggiunto ({n} {var} max)","Limite del gruppo raggiunto ({n} {var} max)",["elementi","elemento"]],multipleSeparator:", ",selectAllText:"Seleziona Tutto",deselectAllText:"Deseleziona Tutto"}});

cps/static/js/libs/bootstrap-select/defaults-ja.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"\u9078\u629e\u3055\u308c\u3066\u3044\u307e\u305b\u3093",noneResultsText:"'{0}'\u306f\u898b\u3064\u304b\u308a\u307e\u305b\u3093",countSelectedText:"{0}/{1} \u9078\u629e\u4e2d",maxOptionsText:["\u9078\u629e\u4e0a\u9650\u6570\u3092\u8d85\u3048\u3066\u3044\u307e\u3059(\u6700\u5927{n}{var})","\u30b0\u30eb\u30fc\u30d7\u306e\u9078\u629e\u4e0a\u9650\u6570\u3092\u8d85\u3048\u3066\u3044\u307e\u3059(\u6700\u5927{n}{var})",["\u30a2\u30a4\u30c6\u30e0","\u30a2\u30a4\u30c6\u30e0"]],selectAllText:"\u5168\u3066\u9078\u629e",deselectAllText:"\u9078\u629e\u3092\u30af\u30ea\u30a2",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-km.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"\u1798\u17b7\u1793\u1798\u17b6\u1793\u17a2\u17d2\u179c\u17b8\u1794\u17b6\u1793\u1787\u17d2\u179a\u17be\u179f\u179a\u17be\u179f",noneResultsText:"\u1798\u17b7\u1793\u1798\u17b6\u1793\u179b\u1791\u17d2\u1792\u1795\u179b {0}",countSelectedText:function(e,t){return"{0} \u1792\u17b6\u178f\u17bb\u178a\u17c2\u179b\u1794\u17b6\u1793\u1787\u17d2\u179a\u17be\u179f"},maxOptionsText:function(e,t){return[1==e?"\u1788\u17b6\u1793\u178a\u179b\u17cb\u178a\u17c2\u1793\u1780\u17c6\u178e\u178f\u17cb ( {n} \u1792\u17b6\u178f\u17bb\u17a2\u178f\u17b7\u1794\u179a\u1798\u17b6)":"\u17a2\u178f\u17b7\u1794\u179a\u1798\u17b6\u1788\u17b6\u1793\u178a\u179b\u17cb\u178a\u17c2\u1793\u1780\u17c6\u178e\u178f\u17cb ( {n} \u1792\u17b6\u178f\u17bb)",1==t?"\u178a\u17c2\u1793\u1780\u17c6\u178e\u178f\u17cb\u1780\u17d2\u179a\u17bb\u1798\u1788\u17b6\u1793\u178a\u179b\u17cb ( {n} \u17a2\u178f\u17b7\u1794\u179a\u1798\u17b6\u1792\u17b6\u178f\u17bb)":"\u17a2\u178f\u17b7\u1794\u179a\u1798\u17b6\u1780\u17d2\u179a\u17bb\u1798\u1788\u17b6\u1793\u178a\u179b\u17cb\u178a\u17c2\u1793\u1780\u17c6\u178e\u178f\u17cb ( {n} \u1792\u17b6\u178f\u17bb)"]},selectAllText:"\u1787\u17d2\u179a\u17be\u179f\u200b\u1799\u1780\u200b\u1791\u17b6\u17c6\u1784\u17a2\u179f\u17cb",deselectAllText:"\u1798\u17b7\u1793\u1787\u17d2\u179a\u17be\u179f\u200b\u1799\u1780\u200b\u1791\u17b6\u17c6\u1784\u17a2\u179f",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-nl.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Niets geselecteerd",noneResultsText:"Geen resultaten gevonden voor {0}",countSelectedText:"{0} van {1} geselecteerd",maxOptionsText:["Limiet bereikt ({n} {var} max)","Groep limiet bereikt ({n} {var} max)",["items","item"]],selectAllText:"Alles selecteren",deselectAllText:"Alles deselecteren",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-pl.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,n){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return n(e)}):"object"==typeof module&&module.exports?module.exports=n(require("jquery")):n(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Nic nie zaznaczono",noneResultsText:"Brak wynik\xf3w wyszukiwania {0}",countSelectedText:"Zaznaczono {0} z {1}",maxOptionsText:["Osi\u0105gni\u0119to limit ({n} {var} max)","Limit grupy osi\u0105gni\u0119ty ({n} {var} max)",["elementy","element"]],selectAllText:"Zaznacz wszystkie",deselectAllText:"Odznacz wszystkie",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-ru.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"\u041d\u0438\u0447\u0435\u0433\u043e \u043d\u0435 \u0432\u044b\u0431\u0440\u0430\u043d\u043e",noneResultsText:"\u0421\u043e\u0432\u043f\u0430\u0434\u0435\u043d\u0438\u0439 \u043d\u0435 \u043d\u0430\u0439\u0434\u0435\u043d\u043e {0}",countSelectedText:"\u0412\u044b\u0431\u0440\u0430\u043d\u043e {0} \u0438\u0437 {1}",maxOptionsText:["\u0414\u043e\u0441\u0442\u0438\u0433\u043d\u0443\u0442 \u043f\u0440\u0435\u0434\u0435\u043b ({n} {var} \u043c\u0430\u043a\u0441\u0438\u043c\u0443\u043c)","\u0414\u043e\u0441\u0442\u0438\u0433\u043d\u0443\u0442 \u043f\u0440\u0435\u0434\u0435\u043b \u0432 \u0433\u0440\u0443\u043f\u043f\u0435 ({n} {var} \u043c\u0430\u043a\u0441\u0438\u043c\u0443\u043c)",["\u0448\u0442.","\u0448\u0442."]],doneButtonText:"\u0417\u0430\u043a\u0440\u044b\u0442\u044c",selectAllText:"\u0412\u044b\u0431\u0440\u0430\u0442\u044c \u0432\u0441\u0435",deselectAllText:"\u041e\u0442\u043c\u0435\u043d\u0438\u0442\u044c \u0432\u0441\u0435",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-sv.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Inget valt",noneResultsText:"Inget s\xf6kresultat matchar {0}",countSelectedText:function(e,t){return 1===e?"{0} alternativ valt":"{0} alternativ valda"},maxOptionsText:function(e,t){return["Gr\xe4ns uppn\xe5d (max {n} alternativ)","Gr\xe4ns uppn\xe5d (max {n} gruppalternativ)"]},selectAllText:"Markera alla",deselectAllText:"Avmarkera alla",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-tr.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,i){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return i(e)}):"object"==typeof module&&module.exports?module.exports=i(require("jquery")):i(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"Hi\xe7biri se\xe7ilmedi",noneResultsText:"Hi\xe7bir sonu\xe7 bulunamad\u0131 {0}",countSelectedText:function(e,i){return"{0} \xf6\u011fe se\xe7ildi"},maxOptionsText:function(e,i){return[1==e?"Limit a\u015f\u0131ld\u0131 (maksimum {n} say\u0131da \xf6\u011fe )":"Limit a\u015f\u0131ld\u0131 (maksimum {n} say\u0131da \xf6\u011fe)","Grup limiti a\u015f\u0131ld\u0131 (maksimum {n} say\u0131da \xf6\u011fe)"]},selectAllText:"T\xfcm\xfcn\xfc Se\xe7",deselectAllText:"Se\xe7iniz",multipleSeparator:", "}});

cps/static/js/libs/bootstrap-select/defaults-zh_Hans_CN.min.js

@@ -0,0 +1,8 @@
+/*!
+ * Bootstrap-select v1.13.14 (https://developer.snapappointments.com/bootstrap-select)
+ *
+ * Copyright 2012-2020 SnapAppointments, LLC
+ * Licensed under MIT (https://github.com/snapappointments/bootstrap-select/blob/master/LICENSE)
+ */
+
+!function(e,t){void 0===e&&void 0!==window&&(e=window),"function"==typeof define&&define.amd?define(["jquery"],function(e){return t(e)}):"object"==typeof module&&module.exports?module.exports=t(require("jquery")):t(e.jQuery)}(this,function(e){e.fn.selectpicker.defaults={noneSelectedText:"\u6ca1\u6709\u9009\u4e2d\u4efb\u4f55\u9879",noneResultsText:"\u6ca1\u6709\u627e\u5230\u5339\u914d\u9879",countSelectedText:"\u9009\u4e2d{1}\u4e2d\u7684{0}\u9879",maxOptionsText:["\u8d85\u51fa\u9650\u5236 (\u6700\u591a\u9009\u62e9{n}\u9879)","\u7ec4\u9009\u62e9\u8d85\u51fa\u9650\u5236(\u6700\u591a\u9009\u62e9{n}\u7ec4)"],multipleSeparator:", ",selectAllText:"\u5168\u9009",deselectAllText:"\u53d6\u6d88\u5168\u9009"}});