#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""This is a form for archiving albums.
An album (in Unibas terminology) is a usually mass-produced physical medium
like an audio CD, video DVD, data CD or DVD (e.g. game CD-ROM),
paper book, vinyl LP, MC, VHS cassette.
This form provides direct or indirect access to the content and almost all
of the metadata concerning the album. This includes
the product description, packaging (single or package of several CDs/DVDs etc.),
contents (document) titles, authors/actors/speakers and other artists (TODO),
of the original and possible copies.
In this form, the user can see if there are copies, in which format,
and initiate copies directly for some formats (e.g. MP3, OGG Vorbis, etc.)
For others, external copy and compression tools are needed,
but the user can archive the resulting files with this form.
TODO:
- Export CD as series of MP3 titles
- with index.html incl. metadata, with cover images
- document.qual field, smallint, 100=same as orig, 0=nosignal only noise, <0 special values
- protocol to exchange interesting docs among peers
- Export to FreeDB, i.e. for all audio CD albums:
if we have track titles (for some albums we have only the album title):
check if the album is already known in Free DB
if not, export the album and track titles to FreeDB
- Fix the following bug:
packageId None
[1594558216L, 8, 150, 21198, 47350, 79929, 105994, 123559, 154445, 184183, 2825]
Traceback (most recent call last):
File "albumform.py", line 611, in checkAudioCd
res = self.readFreeDb()
File "albumform.py", line 577, in readFreeDb
(read_status, info) = CDDB.read(query_info['category'], query_info['disc_id'])
File "/usr/lib/pymodules/python2.6/CDDB.py", line 92, in read
header[0] = string.atoi(header[0])
File "/usr/lib/python2.6/string.py", line 403, in atoi
return _int(s, base)
ValueError: invalid literal for int() with base 10: '<!DOCTYPE'
- React reasonably on double packaging (hierarchy of packages as in Jazz Archive)
- Ask if part or whole backup should be done ONLY IF there is no backup yet
(if there is partial backup, offer to backup the other parts)
- On Set Owner, select album or package, but not series.
- Double-layer progress bar when copying tracks from a CD
- Orig=true if lender is library?
- Preserve/create id3tags; also for already archived MP3s:
add from DB data to MP3 file
- if a product in a series has several objects and user selects one, show that one (prod.1954)!
- update display after making copies etc.
- check EANs with http://www.barcodeisland.com/ (in Postgres 9: http://www.postgresql.org/docs/current/static/isn.html)
select id,length(ean), ean, checksum_ean(substr(ean,1,length(ean)-1)::numeric), length(artno), artno, descr from product where ean IS NOT NULL AND EAN!='' and length(ean)!=13 order by id;
many 11-digit "EANs" are UPC-A 11 digits, printed without checksum, for example
09020400078 which has checksum 4 giving a complete 090204000784
upcdatabase.com
From Smarty's Barcode Scanner
The standard for EANs/UPCs/barcodes now is EAN13, i.e. incl. checksum digit,
used by amazon.de and alphamusic.de and presumably most others.
From 9020400078 we get 090204000784 etc; this may most EANs could be standardized; remain only:
# select id,rt, ean, name from product where ean IS NOT NULL AND EAN!='' and length(ean)!=13 order by id;
See also: http://www.adams1.com/faq.html#upclist
http://www.ean-search.org/perl/ean-search.pl?q=0090204016839
http://www.rebuy.de/i,120197/cd/hans-zonatelli-wassermusik-suite-1-3?ga_source=belboon_rebuy&ga_medium=AFF&ga_content=csv&ga_term=Hans+Zanotelli+-+Wassermusik%2C+Suite+1-3+CD&ga_campaign=belboon_rebuy_AFF_csv_Hans+Zanotelli+-+Wassermusik%2C+Suite+1-3+CD?belboon=01af41070ab002c96e003d9f,2361846,subid=1469064719
gepir.gs1.org
http://www.adams1.com/isbn.html
To check and reformat standard numbers:
http://arthurdejong.org/python-stdnum/
Format is maybe more than mimetype:
- DVD directory, <4.7GB
- DVD directory, >4.7GB
- splittable MP3?
class Format
TODO: Represent albumList by a class, see fillFields().
TODO: Represent each step by a class, e.g. Ripper, Encoder, Tagger, ...
The different types (of e.g. encoders) then are subclasses.
TODO: Aids to handle extras in a DVD, such as "Augusta Mozarteum" in "Eine kleine Zauberflöte".
This is a video, but usually not in a one-file archivable digital form,
and implicitly part of the root document but not a version of it.
The original is represented as a video document (mimetype 112: MP)
and its archivable version is an MPG file.
The original needs a cp entry. Its versionof field is empty.
So it is displayed as a document contained in the album,
not a version of its root doc.
(Although implicitly it is of course part of the root doc.)
Suggest
- find album
- function archivePartDoc():
- let user select file
- archive it as in newDocumentFromFile
- add origdoc and origdoc-cp entries
A book with CD can be represented as a package of ot="book with CD"
containing a book (which could be scanned) and a CD (which can be
archived as one or more MP3 files).
(TODO: export such structure diagrams with connectiontree, as graphics or text)
Inlets of CDs may be handled like this as well,
or you deal with them just as documents.
In this case... (TODO) ... similar to front image?
Steps etc. described in node 7109. Summary:
Sources are: Given: file data
- CD/DVD, external source no no
- Preripped files on HD yes no
- The archive yes yes
- Archive data only (theoretical) no yes
Destination can be:
- single document
- work
- album (root doc)
- package (complete package in one document, not implemented)
- series (complete package in one document, not implemented)
Steps:
- Get basic metadata from source (e.g. FreeDB, NOT from infosource yet)
- Show album tree hierarchy
- Fetch data from infosource
- Add data from infosource table to handled entities (products, documents etc.)
- Tag document files with metadata (id3 tagging etc.)
- Schedule action
- Rip from disc (only for source = CD or DVD)
- Regroup (split/combine)
- Encode (convert)
- Store: copy/move to destination
- Calculate hash
- Delete (data and file)
Album Form proposes next step.
Some actions can also be performed in other forms, e.g. document form.
TODO: Find a standard notation for DB relationships, similar to flow diagrams,
for graphical documentation.
For editing:
several entries can be selected
at the same time, and are given container document or artist or whatever
information.
Part - whole relation doesn't go via "version", but directly.
I.e., if there are 4 tracks on a CD for the parts of Beethoven's Eroica,
then they can be selected and be made parts of a container document,
that has no copy, and orchestra, director etc. is associated with this,
and composer as well. Composer is also associated with
the composition as such (MusicXML or whatever), so that's redundant,
but doesn't matter. Often the "composition as such" is not available,
so we need this container document.
The "composition as such" is a good candidate for an
<ulink url="node3577">abstract document</ulink>.
License of DVD copy can be:
- "copy protected", if original is copy protected and you decide
that the copy's license should match the original's license
- "fair use", if the (legal, formal) copy protection does not
apply to you(r country) or you simply decide to ignore it
- "shareware" for some DVDs released as shareware (promotion)
Another decision is what to do when sharing documents (files) and data.
That can depend on mimetype as well.
To automatize repetitive tasks it is necessary the computer
makes decisions for the user; but sometimes it needs
the user's decision. Asking the user too often wastes his time.
Asking him to rarely increases the risk of wrong decisions.
If a decision is to be taken (by the program or the user),
the program first collects facts, finds out what's to decide,
which are the options, which is the most likely option.
Also it reads settings about how to decide, in which cases
to ask the user.
Based on these, it decides whether to ask the user or not.
If not, it takes the option that has been identified as the most
likely one.
If the user needs to be asked, the program first prepares
a message text (HTML) that states what is to decide and
which options are given. (See examples in filecheck.py)
One button per option is displayed. The most likely option
is marked as default. Also a checkbox "Always decide this way"
is displayed. If the user checks it, then in this series of
decisions he is not asked again.
If he clicks Cancel, the current operation is interrupted,
and if this is part of a series, the user is asked whether
to cancel altogether. If yes, the series is abandoned altogether.
If no, the next operation in the series is performed.
(This can maybe be cascaded. If the series is abandoned and
is part of a meta-series, an outer loop, the user is asked
whether to abandon the outer loop as well, and so on.)
In any case, atomic operations that can't be completed
must be rolled back.
Genre can be a notion, and rel:
Rock'n'Roll is genre of "Rock Around the Clock"
TODO: Einheitliche Namensgebung für Nummern, Indexe, Anzahlen
Nummer: number (French: numéro) z.B. episode Nr., not necessarily consecutive, there may be holes in numbering
number (French: nombre), count or quantity (German: Anzahl), how many, prop.: qt
index: from 0 or 1 to qt or qt-1
What is counted:
s: series
A series contains packages (episodes), but it may be a null series.
In that case it is only one episode/package and will not be mentioned
in the hierarchy.
p: package
A package contains albums (products with objects), but it may be a null package,
i.e. a single album. In that the package level and will not be mentioned
in the hierarchy.
a: album
An album, i.e. at least an object, must exist, even if there is no product.
i: side
w: work
d: document
Main idea: If a level has only one element, then the level is not mentioned.
There is a class for each level, and for the element of each level.
The level class object provides:
- getName(): a name (which can be derived from the element of the upper level)
- getId(), may be None if qt<=1
- getQuantity(), quantity/count
if qt=0, there is nothing to be displayed, neither in this nor in lower levels
if qt=1, this level is null but contains lower levels, only those are displayed
(except for album/object and document level, these are always displayed)
if qt>=2, this level is displayed and the lower levels
if there is an album, the qts for series and package (and album) are >=1
- getElements(), an element yielder (factory?), yields exactly qt elements
- if there is no series, a dummy 1-element series will be constructed,
same with package (and side and work?)
- a progress bar?
The level element object provides:
- a name
- an index (0 or 1 to qt)
- as members all needed variables, e.g. for series: eObjectId, eProductId
"""
__author__ = "Volker Paul"
__copyright__ = "Copyright 2011, Volker Paul"
__license__ = "GPL 2"
__maintainer__ = "Volker Paul"
__email__ = "volker.paul@v-paul.de"
__status__ = "Development"
__docformat__ = 'restructuredtext'
import logging, sys, os, os.path, copy, re, textwrap, urllib2
from copy import deepcopy
import Levenshtein
import CDDB, DiscID
from PyQt4.QtCore import *
from PyQt4.QtGui import *
from operator import itemgetter
import pprint
from .utilities import *
from . import document, product, object, infosource, mimetype, person
from . import db, dbq, archive, settings, instancewidget, finddialog
from . import metadatareader, connectiontree, tools, progress, dvdinfo
import entityelement
from functools import reduce
importParentDirList = ["CD","LP","DVD","MC","book"]
def getEpisodeObject(episodeId, ot):
"""Return tuple (objectId, productId) from given episode document ID."""
origAlbumSql = textwrap.dedent("""
SELECT o.id, productid FROM object o, cp
WHERE cp.documentid=%s AND cp.objectid=o.id AND o.orig
""")[1:-1] % episodeId
origAlbumDicList = db.dicList(origAlbumSql)
if origAlbumDicList:
return (origAlbumDicList[0]["id"], origAlbumDicList[0]["productid"])
smtSql = textwrap.dedent("""
SELECT o.id, productid FROM object o, cp
WHERE cp.documentid=%s AND cp.file='/' AND cp.objectid=o.id AND ot~*'%s'
""")[1:-1] % (episodeId, ot) # Same Medium Type
smtDicList = db.dicList(smtSql)
if smtDicList:
return (smtDicList[0]["id"], smtDicList[0]["productid"])
return (None, None)
def getEpisodeList(seriesId, documentId=None):
"""Return list of documents (as dictionaries) that are episodes of given series,
sorted by order in series."""
if not seriesId: return []
addCrit = "AND e.id=%s" % documentId if documentId else ""
episodesSql = textwrap.dedent("""
SELECT rel.ord, e.id, e.title FROM document e, rel
WHERE relid=63832 AND rel.ent1=5 AND rel.id1=e.id AND rel.ent2=5 AND rel.id2=%s
%s
ORDER BY ord;
""")[1:-1] % (seriesId, addCrit) # SQL to get episode documents
#print "episodesSql:", episodesSql
#print "getEpisodeList", db.dicList(episodesSql)
return db.dicList(episodesSql)
def getEpisodeObjectList(seriesId, mainOt, documentId=None):
"""Return episodeList of for all episodes of given series with object ID.
An episode is represented by the triple (ord, objectId, poductId).
Episodes without object get listed in the warning string.
"""
res = []; noMediumDoc = ''
for e in getEpisodeList(seriesId, documentId):
(oid, pid) = getEpisodeObject(e["id"], mainOt)
if oid:
res.append((int(e["ord"]) or 0, oid, pid))
else:
noMediumDoc += "%s: %s\n" % (e["id"], e["title"])
if noMediumDoc:
msgbox("There are episode documents without album:\n%s" % limitLines(noMediumDoc))
return res
class Series():
"""Represents a series."""
def __init__(self, objectId, productId, seriesId, mainOt):
self.episodeList = []
if objectId and not seriesId:
rootDocId = db.getRootDocId(objectId)
seriesId = dbq.getSeriesId(rootDocId)
# print "seriesId", seriesId
self.seriesId = seriesId
if seriesId:
# seriesItem = self.regItem(None, "series", db.descr("document", seriesId), id=seriesId)
if objectId:
msg = "This is an episode of a series. Show other episodes too?"
if QMessageBox.question(None, "Unibas AlbumForm", msg,
QMessageBox.StandardButtons(QMessageBox.No | QMessageBox.Yes),
QMessageBox.No) == QMessageBox.Yes:
self.episodeList = getEpisodeObjectList(seriesId, mainOt)
else: # Show only given episode.
self.episodeList = getEpisodeObjectList(seriesId, mainOt, rootDocId)
else: # Show all episodes.
self.episodeList = getEpisodeObjectList(seriesId, mainOt)
else: # No series, episodeList has only one element.
self.episodeList = [(0, objectId, productId)]
def getId(self):
return self.seriesId
def getElements(self):
self.progress = None
if self.getQuantity() > 1:
self.progress = QProgressDialog("Preparing display of series:\n" + self.getName(),
"Cancel", 0, self.getQuantity(), None)
self.progress.setWindowTitle("Unibas AlbumForm")
self.progress.setMinimumWidth(400)
self.progress.setWindowModality(Qt.WindowModal)
self.progress.setMinimumDuration(1)
self.progress.setAutoClose(True)
for i, (c, o, p) in enumerate(self.episodeList):
if self.progress: self.progress.setValue(i)
if self.progress and self.progress.wasCanceled(): break
yield SeriesElement(c, o, p)
if self.progress: self.progress.reset()
def getQuantity(self):
return len(self.episodeList)
def getName(self):
#if not self.seriesId: return None
#return db.readSql("SELECT title FROM document WHERE id=" + self.seriesId)
return db.descrOrNone("document", self.seriesId)
class SeriesElement():
def __init__(self, cnt, objectId, productId):
self.cnt, self.objectId, self.productId = cnt, objectId, productId
class Package():
"""Represents a package which can contain one or more albums (objects).
Replaces dbq.getObjectProductList().
Internal storage: opl (object-product-list)
is a list of tuples (objectId, productId)
where the tuple with index 0 is the package (may be (None, None))
and the following tuples represent albums.
$ python unibas_cli.py -c "import unibas.albumform; pa = unibas.albumform.Package('11141'); print pa.getObject()"
<object 11141: Mozart: Salzburg Symphonies, Eine kleine Nachtmusik>
"""
def __init__(self, ge, productId=None):
"""Input may be an album (object and/or product),
GenericEntity (object|product) or package product.
At least one of objectId, productId must be not None.
However, for each product only one album is returned.
That album is objectId if it is given, otherwise an original,
if there is. The list is ordered by object.descr.
"""
self.opl = [] #(None, None), (None, None)] # package and first album
if not ge and not productId: return
if isinstance(ge, entityelement.GenericEntity):
if ge.entityType == "object":
objectId = ge.getId() #; productId = None
else:
productId = ge.getId(); objectId = None
else:
objectId = ge
#print "objectId, productId", objectId, productId
#log = logging.getLogger("getObjectProductList")
#log.debug("objectId: %s, productId: %s" % (objectId, productId))
# Decide if it is a package or album.
# - no product ID given and none in DB -> album
# - product ID given and has prods -> package
# - product ID given and has packageId -> album
if not productId:
productId = db.readSql("SELECT productid FROM object WHERE id=%s"
% objectId, 0, 1)
sqlBase = "SELECT id FROM product p WHERE p.packageid=%s"
prods = db.valueList(sqlBase % productId) if productId else None
# log.debug("prods: %s" % unicode(prods))
if not prods: # Product is not package but still may be part of one.
# if not productId: return [(0, None, None), (1, objectId, productId)]
if not productId:
self.opl = [(None, None), (objectId, None)]
return
packageId = db.readSql("SELECT packageid FROM product WHERE id=%s"
% productId, 0, 1)
if not packageId: # Definitely not a package.
if not objectId: # Get object by productId; prefer originals.
sql = "SELECT id FROM object WHERE productid=%s ORDER BY orig DESC LIMIT 1" % productId
dl = db.dicList(sql)
objectId = dl[0]["id"] if dl else None
self.opl = [(None, None), (objectId, productId)]
return
prods = db.valueList(sqlBase % packageId) # is part of package, now select package's parts
packageObjectId = db.readSql("SELECT id FROM object WHERE productid=%s" % packageId, 0, 1)
self.opl = [(packageObjectId, packageId)]
else: # productId is package
self.opl = [(objectId, productId)]
self.packageId = self.getId()
objectList = []
# Loop over package parts (products), one entry per product,
# and select (at most) one object per product,
# preferably the given objectId or an original.
for productId in prods: # loop over package parts
#sql = "SELECT id, descr FROM object WHERE productid=%s" % productId
#sql += " ORDER BY (id=%s) DESC, orig DESC;" % (objectId or 0)
##print sql
## Ordering should ensure that given objectId is first (if given),
## otherwise an original. Only dl[0], i.e. the first, is taken.
#dl = db.dicList(sql)
#if dl: objectList.append((productId, dl[0]["id"], dl[0]["descr"]))
(oid, descr) = dbq.getOrigObjectOfProduct(productId, objectId)
if oid: objectList.append((productId, oid, descr))
# Sort objectList by descr which has index 2 in the tuples.
objectList.sort(key=lambda x: x[2])
for o in objectList:
self.opl.append((o[1], o[0]))
def getId(self):
return self.getProductId(0)
def getElements(self):
self.progress = None
if self.getQuantity() > 1:
self.progress = QProgressDialog("Preparing display of package:\n" + self.getName(),
"Cancel", 0, self.getQuantity(), None)
self.progress.setWindowTitle("Unibas AlbumForm")
self.progress.setMinimumWidth(400)
self.progress.setWindowModality(Qt.WindowModal)
self.progress.setMinimumDuration(1)
self.progress.setAutoClose(True)
for i in range(1, len(self.opl)):
if self.progress: self.progress.setValue(i-1)
if self.progress and self.progress.wasCanceled(): break
yield PackageElement(i, self.opl[i][0], self.opl[i][1])
if self.progress: self.progress.reset()
def getQuantity(self):
return len(self.opl)
def getName(self):
return db.descr("product", self.getId()) if self.getId() else ''
#def getAlbumCount(self):
#return len(self.opl) - 1
def getObjectId(self, i=1):
#if len(self.opl) < i + 1: return None
#return self.opl[i][0]
return self.opl[i][0] if len(self.opl) >= i + 1 else None
def getProductId(self, i=1):
return self.opl[i][1] if len(self.opl) >= i + 1 else None
# TODO: Maybe restructure this class, decide which methods are needed.
#def getPackageId(self):
#return self.getProductId(0)
#def getPackageObjectId(self):
#return self.getObjectId(0)
#def getObject(self, i=1):
#"""Return object which may be empty (no ID),
#but still an object, not None."""
#o = object.Object(self.getObjectId(i))
#o.readFromDb()
#return o
#def getProduct(self, i=1):
#"""Return product which may be empty (no ID),
#but still a product, not None."""
#p = product.Product(self.getProductId(i))
#p.readFromDb()
#return p
#def getPackage(self):
#return self.getProduct(0)
#def getPackageObject(self):
#return self.getObject(0)
#def getOPList(self):
#"""For compatibility with getObjectProductList()."""
#return [(i, op[0], op[1]) for (i, op) in enumerate(self.opl)]
#def getObjectDic(self, i=1):
#return getEntityDic("object", self.getObjectId(i))
#def getProductDic(self, i=1):
#return getEntityDic("product", self.getProductId(i))
def getVersionMimetypeIds(self, i=1):
objectId = self.getObjectId(i)
if not i: return []
versionSql = "SELECT DISTINCT * FROM v_albumversion WHERE albumid=" + objectId
versionDocs = db.dicList(versionSql)
ids = [d["copytypeid"] for d in versionDocs]
return ids
class PackageElement():
def __init__(self, cnt, objectId, productId):
self.cnt, self.objectId, self.productId = cnt, objectId, productId
def getObjectDic(self):
return getEntityDic("object", self.objectId)
def getProduct(self):
"""Return product which may be empty (no ID),
but still a product, not None."""
p = product.Product(self.productId)
#if not p.getId(): return p
if self.productId: p.readFromDb()
return p
def getMaintype(self):
if not self.productId: return ''
p = product.Product(self.productId)
return (p.getValue("rt") or '').lower()
def getEntityDic(entityType, id):
"""If id is a valid ID and references an entityType record,
return this record as dic, else None."""
if not id: return None
if int(getId(id)) <= 0: return None
l = db.dicList("SELECT * FROM %s WHERE id=%s" % (entityType, id))
return l[0] if l else None
def getText(item, col):
if not item: return None
return unicode(item.text(col))
def getNumOr1(item, col):
if not item: return 1
text = unicode(item.text(col)).strip()
if not text: return 1
return int(text)
def traverseRecursive(item, level=0):
"""Yield item if level>0 and all children.
"""
if level>0: yield item
for i in range(item.childCount()):
for res in traverseRecursive(item.child(i), level+1): yield res
# Docs -> DocsToArchive, docsToA; DocsInArchive, docs1a
# similar access patterns for DocsToArchive and DocsInArchive
class DocsInArchive():
"""Represents the archived documents that are associated
with a given album or package.
Useful for:
- exporting
- reformatting
- reproducing the album
- checking if an album and its documents are already
(and completely) archived, see checkBarcode()
See document.AlbumDocs()
"""
def __init__(self):
pass
class DocsToArchive(): # docsToA = DocsToArchive()
"""All documents (origs and copies) of an album or package.
Album index counts from 1, and is 1 by default.
Replaces albumList.
albumList was a list of albums where
each album is a list of (orig,copy) document tuples
"""
def __init__(self):
self.orig = {} # index -> list of Document()s
self.copy = {} # index -> list of Document()s
self.maxIndex = 0 # number of albums in package
def pprint(self):
pp = pprint.PrettyPrinter()
print "------------------ orig:"
pp.pprint(self.orig)
print "------------------ copy:"
pp.pprint(self.copy)
def getOrig(self, index=1):
"""Return list of orig Document()s in given album, default 1."""
return self.orig.get(index) or []
def getCopy(self, index=1):
"""Return list of copy Document()s in given album, default 1."""
return self.copy.get(index) or []
def getAlbumCount(self):
"""Return number of albums in package."""
return maxIndex
def getDocCount(self, index):
orig = self.getOrig(index)
if not orig: return 0
return len(orig)
def setPair(self, albumIndex, pair):
"""Set orig of given album to given orig, same with copy."""
self.orig[albumIndex] = [pair[0]]
self.copy[albumIndex] = [pair[1]]
def ensureAlbumExists(self, albumIndex):
if not self.orig.get(albumIndex): self.orig[albumIndex] = []
if not self.copy.get(albumIndex): self.copy[albumIndex] = []
def addPair(self, albumIndex, pair):
"""Add given orig to origs of given album, same with copy."""
self.ensureAlbumExists(albumIndex)
self.orig[albumIndex].append(pair[0])
self.copy[albumIndex].append(pair[1])
def albumList2Docs(self, albumList):
# print "albumList", albumList
self.maxIndex = len(albumList)
for albumIndex in range(self.maxIndex):
# print "albumIndex", albumIndex
self.orig[albumIndex+1] = []
self.copy[albumIndex+1] = []
docTuples = albumList[albumIndex]
for docTupleIndex in range(len(docTuples)):
docTuple = docTuples[docTupleIndex]
self.orig[albumIndex+1].append(docTuple[0])
self.copy[albumIndex+1].append(docTuple[1])
def docList_i(self, i):
return (self.getOrig(i), self.getCopy(i))
def getAlbumDocPair(self, albumCnt, level, cnt):
"""Return tuple (orig, copy) of single documents."""
index = 0 if level == "album" else int(cnt)-1
#(orig, copy) = (self.getOrig(albumCnt)[index],
#self.getCopy(albumCnt)[index])
(orig, copy) = (lget(self.getOrig(albumCnt), index),
lget(self.getCopy(albumCnt), index))
#else:
#(orig, copy) = (self.getOrig(albumCnt)[int(cnt)-1],
#self.getCopy(albumCnt)[int(cnt)-1])
return (orig, copy)
# replace:
# docList[i] if len(docList)>=i+1 else (None,None)
# extdocs = albumList[cnt-1] if cnt<=len(albumList) else [] # cnt is 1-based
class AlbumHierarchy(QTreeWidget):
"""Widget that represents the album hierarchy tree structure.
"""
def __init__(self, parent=None, name=None):
QTreeWidget.__init__(self)
self.createContextMenu()
# TODO: Determine device if necessary.
self.device = None #settings.getSetting("Devices/dvdwriter") or None
self.lastImportedType = None # old ot, so next dir import proposal will be of this type
self.baseColumnWidths = None
#self.lastMediumType = '' # used as default ot in addPackage
self.lastPackageDataset = ('', '', 'CD', 2)
self.clear() # defines and clears some variables
# self.albumDirList): # in importDirectory(), possibly 1-element album list, set by setImportPackage()
# h.defaultFormats is a dictionary maintype=>list of formatIds;
# For the first of these formats, a copy per doc is scheduled by default if it does not exist yet.
# TODO: Make customizable in settings, but offer only supported formats (mimetype.support)
# select id, type, split_part(type,'/',1) from mimetype where support>0
self.defaultFormats = {"audio":['57','59','165','163'],
"video":['133','125','104','118','108','168','105'],
"text":['1'], "application":['130']}
self.backgroundColor = {
"series": "lightgrey",
"package": "cyan",
"album": "lightblue",
"side": "lightgreen",
"work": "yellow",
"document": "white"
}
QObject.connect(self, SIGNAL("currentItemChanged (QTreeWidgetItem *,QTreeWidgetItem *)"),
self.currentItemChanged)
QObject.connect(self, SIGNAL("itemCollapsed (QTreeWidgetItem *)"), self.itemCollapsed)
QObject.connect(self, SIGNAL("itemExpanded (QTreeWidgetItem *)"), self.itemExpanded)
def clear(self):
h = self
self.ean = None # see checkDisc
self.mediaid = None
self.dvdSize = None
self.objectId = None
self.maintype = None # "audio", "video" etc. for CD/DVD in drive
# in setColWidths(), this determines formats via self.defaultFormats
self.ot = None # object type, e.g. "CD","DVD","UnDVD","book","MC","LP"
self.albumTitle = None
self.invisibleRootItem().takeChildren()
# These two tell in which mode AlbumForm currently is;
# if self.discid: archiving a CD/DVD
# if dirToImport: importing a directory
# else: cleared or showing archived data; TODO: put these in __init__
self.discid = None # audio CD (FreeDB) or self-calculated discid for CD/DVD in drive
self.dirToImport = None # in and after importDirectory, the directory to import
# self.filesToArchive = None # dic documentId->file (or subdir) from importDirectory (obsolete?)
self.objectInDevice = None # ID of object in above device
self.formats = [] # list of supported mimetype.Mimetype()
self.formatColumn = {} # keys: mimetype IDs; values: column numbers
self.setSelectionMode(QAbstractItemView.ExtendedSelection)
self.defineBaseColumns()
self.setHeaderLabels(self.baseColumnNames) #["Level","Title"])
# self.lastImportedType is NOT cleared!
def defineBaseColumns(self):
c=0; self.levelColumn = c # indicates level (package, album, track, ...)
c+=1; self.curColumn = c # marks current album of a package
c+=1; self.rnColumn = c # running number per parent, e.g. doc per work
c+=1; self.dnColumn = c # document counter per album object
c+=1; self.dbTitleColumn = c # title from DB
c+=1; self.extTitleColumn = c # title from external infosource
c+=1; self.idColumn = c # id (of product, document etc.)
c+=1; self.fileColumn = c # filename (without directory)
c+=1; self.albumColumn = c # tells number of albums, only in album item
c+=1; self.formatColumnStart = c # first copy format column
self.baseColumnCount = self.columnCount = c # For formats, no columns are reserved yet
self.baseColumnNames = ["Level","?","RN","DN","Title from DB","External title","ID","File","Alb"]
if not self.baseColumnWidths:
self.baseColumnWidths = [100,10,30,25,240,100,50,40,30]
def setDevice(self):
"""Set and return the CD/DVD device, in self.device.
If self.device is already set, do nothing.
Otherwise list all devices. If there are several different ones,
let the user select.
"""
if self.device: return self.device
l = []
for cddvd in ("cd","dvd"):
for rw in ("reader","writer"):
l.append(settings.getSetting("Devices/%s%s" % (cddvd, rw)))
s = set(l)
if len(s) == 1:
self.device = l[0]
return self.device
s = StringSelector(list(s))
if not s.exec_():
return None
self.device = s.selection()[0]
return self.device
def currentItemChanged(self):
item = self.currentItem()
docId = self.getDocId(item)
self.emit(SIGNAL("changedDocId"), docId)
objectId = self.getObjectId(self.currentItem())
self.emit(SIGNAL("changedObjectId"), objectId)
#TODO: also show package even if there is no package object
packageId = self.getPackageId(self.currentItem())
print("packageId ", packageId)
if packageId and not objectId:
self.emit(SIGNAL("changedProductId"), packageId)
def itemCollapsed(self):
pass #print "itemCollapsed"
def itemExpanded(self):
item = self.currentItem()
if item: item.setExpanded(True)
def setExpanded(self, b):
"""Set whether item is expanded,
and if it is read from table if necessary.
"""
QTreeWidgetItem.setExpanded(self,b)
# class AlbumHierarchy context menu and its functions
def createContextMenu(self):
"""Create tree context menu."""
self.contextMenu = QMenu()
# addMenuAction(self, self.contextMenu, "&Test", self.test)
addMenuAction(self, self.contextMenu, "Show &info", self.showInfo)
addMenuAction(self, self.contextMenu, "&Show archived version", self.showVersion)
addMenuAction(self, self.contextMenu, "Show external version", self.showExternalVersion)
addMenuAction(self, self.contextMenu, "Show file", self.showFile)
# addMenuAction(self, self.contextMenu, "Select action to schedule", self.scheduleAction)
addMenuAction(self, self.contextMenu, "Show form", self.showForm)
addMenuAction(self, self.contextMenu, "Show Connection Tree", self.showConnectionTree)
def contextMenuEvent(self, e):
self.contextMenu.exec_(QCursor.pos())
def test(self):
pass
#h = self
##print "discid from DB:", self.productWidget.getValue("discid")
#item = self.currentItem()
#level = getText(item, h.levelColumn)
#id = getText(item, h.idColumn)
#file = getText(item, h.fileColumn)
#title = getText(item, h.dbTitleColumn)
#print(level, id, file, title)
#entityName="document"
#if level == "package": entityName="product"
#if level == "album": entityName="object"
#(e,l) = document.docParts(id, entityName)
#if e:
#errmsg(e)
#return
#msgbox(document.getStringEntityList(l))
#return
#item = item.parent()
#print("parent data:")
#level = getText(item, h.levelColumn)
#id = getText(item, h.idColumn)
#file = getText(item, h.fileColumn)
#title = getText(item, h.dbTitleColumn)
#print(level, id, file, title)
#for item in traverseRecursive(self.invisibleRootItem()):
#print(unicode(item.text(h.dbTitleColumn)), ("SELECTED" if item.isSelected() else ''))
def showInfo(self):
"""Show detailed info about current node and relatives,
from KnowledgeTree and node table.
"""
pass
#item = self.currentItem()
#level = getText(item, self.levelColumn)
#id = getText(item, self.idColumn)
#file = getText(item, self.fileColumn)
#title = getText(item, self.dbTitleColumn)
#print(level, id, file, title)
#(e,l) = document.docParts(id, level, title)
#if e:
#errmsg(e)
#return
#s = "<pre>%s</pre>" % document.getStringEntityList(l)
#print(s)
#msgbox(s)
#return
#h = self
#item = self.currentItem()
#albumId = self.getAncestorAlbumId(item)
#msgbox("albumId:", albumId)
#return
#level = getText(item, self.levelColumn)
#id = getText(item, self.idColumn)
#file = getText(item, self.fileColumn)
#title = getText(item, self.dbTitleColumn)
## msg = "level: %s, id: %s, file: %s, title: %s" % (level, id, file, title)
#msg = getItemDescr(h, item) + '\n'
#descs = getDescendantItems(h, item)
#for i in descs:
#msg += getItemDescr(h, i) + '\n'
#msgbox(msg)
def showVersion(self):
"""Show archivable version of item's document."""
docId = self.getDocId(self.currentItem())
if not docId:
errmsg("Cannot show document, no document ID!")
return
# document.showDocumentVersion(docId)
document.showDocumentVersionDialog(docId)
def showExternalVersion(self):
"""Show/playback external version, i.e. from CD/DVD or
from the directory to import."""
item = self.currentItem()
#print "self.ot:", self.ot
#print "self.device:", self.device
#print "self.maintype ", self.maintype
cnt = getText(item, self.rnColumn)
level = getText(item, self.levelColumn)
if self.discid:
ot = otHead(self.ot)
device = settings.getSetting("Devices/%sreader" % ot)
if ot == 'CD' and self.maintype == "audio":
# xine cdda:/[<device>][/<track-number>]
if level == "album": cnt = ''
# cmd = "gxine cdda://" + cnt # TODO: make configurable, and ensure it is available
cmd = "gxine cdda:/{}/{}".format(device, cnt) # for MRL syntax see man 5 xine
# cmd = "mplayer cdda://%s &" % cnt # TODO: make configurable, and ensure it is available
btc(cmd)
return
elif ot == 'DVD':
if level != "album":
msgbox("Only whole DVD can be played")
cmd = "xine dvd:/" # TODO: make configurable, and ensure it is available
btc(cmd)
return
else: #if self.ot == 'CD' and self.maintype == "audio":
# Try to mount; TODO!
#bt("mount /media/cdrom")
#bt("konqueror /media/cdrom &")
bt("mount '%s'" % device)
bt("konqueror '%s' &" % device)
elif self.dirToImport:
if level != "document": # or level == "album":
msgbox("Currently only for documents!")
elif level == "document":
albumItem = self.getAncestorAlbumItem(item)
albumCnt = int(getText(albumItem, self.rnColumn))
docDic = self.albumList[albumCnt-1][int(cnt)-1] # TODO
srcPath = docDic["albumDir"] + '/' + docDic["file"]
btc("vlc '%s'" % srcPath)
else:
msgbox("No CD/DVD and no directory to import")
def showFile(self):
"""TODO"""
file = getText(self.currentItem(), self.fileColumn)
ext = file.split('.')[-1]
m = db.dicList("SELECT * FROM mimetype WHERE ext=%s" % q(ext))
if not m:
errmsg("Can't guess mimetype")
return
dir = getText(self.currentItem().parent(), self.fileColumn)
cmd = "%s '%s/%s' &" % (m[0]["program"], dir, file)
os.system(cmd.encode('utf-8'))
# some AlbumHierarchy helper functions
def getRootDocId(self, item):
"""Return root document ID, i.e. whose cp.file='/', of an album item. May be None."""
#objectId = getText(item, self.idColumn)
id = getText(item, self.idColumn)
level = getText(item, self.levelColumn)
if level == "package":
objectId = dbq.getObjectOfSimpleProduct(id)
else:
objectId = id
print("objectId", objectId)
d = db.getRootDocDic(objectId)
return d["id"] if d else None
def getDocId(self, item):
"""Return document ID of current item.
For series, work and document levels, this is the item's id itself."""
id = getText(item, self.idColumn)
level = getText(item, self.levelColumn)
if level == "package": return getProductRootDocId(id)
if level == "album": # For album items, object ID is stored in id field.
return self.getRootDocId(item)
return id
def getObjectId(self, item):
"""Return object ID of item. For all levels except album and package, return None."""
id = getText(item, self.idColumn)
level = getText(item, self.levelColumn)
if level == "album": return id
if level == "package": return db.getMostCompleteAlbum(id)[0]
return None
def getPackageId(self, item):
"""Return package (product) ID of item if level is package, else None."""
if getText(item, self.levelColumn) == "package":
return getText(item, self.idColumn)
return None
def showForm(self):
"""Show Unibas document form of current item."""
item = self.currentItem()
level = getText(item, self.levelColumn)
id = getText(item, self.idColumn)
if not id:
errmsg("No document")
return
# entityDic = {"package":"product", "album":"product", "work":"document", "document":"document"}
# if not level in entityDic:
# errmsg("Unknown entity for level " + level)
# return
# instancewidget.showEntityForm(self, entityDic[level], id)
instancewidget.showEntityForm(self, "document", id)
def showConnectionTree(self):
"""Show Unibas connction tree of current item."""
item = self.currentItem() or self.getAlbumItem()
if not item: return
level = getText(item, self.levelColumn)
id = self.getDocId(item)
if id:
connectiontree.ConnectionTreeForm(self, "document", id, 'id').show()
return
id = getText(item, self.idColumn)
# if not id: id = self.productWidget.getId()
# if not id: id = self.objectWidget.getId()
if not id:
errmsg("Could not find out ID")
return
entityDic = {"package":"product", "album":"object", "work":"document", "document":"document"}
if not level in entityDic:
errmsg("Unknown entity for level " + level)
return
# instancewidget.showEntityForm(self, entityDic[level], id)
connectiontree.ConnectionTreeForm(self, entityDic[level], id, 'id').show()
def getAncestorAlbumId(self, item):
"""Return ID of given item if it is an album or of
an ancestor if that is an album.
"""
while 1:
if getText(item, self.levelColumn) == "album":
return getText(item, self.idColumn)
item = item.parent()
if not item: return None
def getAncestorAlbumItem(self, item):
"""Return given item if it is an album or
the ancestor that is an album.
"""
while 1:
if getText(item, self.levelColumn) in ("album", "package"): return item
item = item.parent()
if not item: return None
def getProductWithEan(self, item):
"""Return next product with EAN in hierarchy (as dictionary), or None.
"""
while 1:
id = getText(item, self.idColumn)
level = getText(item, self.levelColumn)
if level == "album":
productId = db.readSql("SELECT productid FROM object WHERE id=%s" % id, 0, 1)
if productId:
dl = db.dicList("SELECT * FROM product WHERE id=%s" % productId)
if dl and dl[0]["ean"]: return dl[0]
elif level == "package":
dl = db.dicList("SELECT * FROM product WHERE id=%s" % id)
if dl and dl[0]["ean"]: return dl[0]
item = item.parent()
if not item: return None
def checkBarcode(self):
"""
TODO: first part reused in setProduct()
TODO: Develop Unibas barcode system which codes entity and id
print such barcodes on objects and in rooms
also encode commands such as "speak all objects in this room"
barcode as mediaid -> object -> product
or barcode as ean -> product
check if product has EAN or artno, warn if not
product -> object -> document (root or first) (choose object with most documents)
warn if no object
warn if no document
try to show document
TODO: Handle packages, works, ...
"""
msg = ""
caption = "Check Barcode"
while 1:
msg += "Please enter barcode (EAN or library medium ID) or Object ID"
(barcode, ok) = QInputDialog.getText(None, caption, msg, QLineEdit.Normal)
if not ok: return
msg = ""
barcode = unicode(barcode).strip()
if not barcode:
if not productId: return
print "productId", productId
# self.selectProduct(productId)
print "self.fillFields(productId=%s)" % productId
self.fillFields(productId=productId)
return
objectId = None
if barcode[0] == 'o': # object ID
objectId = barcode[1:]
msg += "Object ID directly given as %s\n" % objectId
else:
res = db.lookup("v_objectowner", {"mediaid":barcode},
["id", "descr"], "%(id)s: %(descr)s")
if res:
msg += "Found via mediaid object %(id)s: %(descr)s\n" % res
objectId = res["id"]
productId = None
if objectId:
productId = db.readSql("SELECT productid FROM object WHERE id=%s" % objectId, 1, 1)
else:
ean13 = "{0:0>13}".format(barcode) # pad with leading zeroes
#print "ean13:", ean13
#product = db.lookup("product", {"ean":ean13}, ["id", "name"])
sql = "SELECT id, name FROM product WHERE lpad(ean, 13, '0')=%s" % q(ean13)
#print sql
dl = db.dicList(sql)
# product = dl[0] if dl else None
#print "product ", product
# productId = product["id"] if product else None
if dl:
productId = dl[0]["id"]
msg += "Found via EAN product %(id)s: %(name)s\n" % dl[0]
if not productId:
msg += "No product found\n"
else:
msg += "Product %s\n" % db.descrWithId("product", productId)
objectId = dbq.getObjectOfSimpleProduct(productId)
if not objectId:
msg += "Product found, but no object for it"
# for packages see docParts
#(e, d) = document.getObjectDocOrList(objectId) #,0)
# ge = entityelement.GenericEntity("object", objectId)
ge = entityelement.GenericEntity("product", productId)
albumdocs = document.AlbumDocs(ge)
(e, d) = albumdocs.getObjectDocOrList()
msg += limitLines(d) + '\n' + limitLines(e)
print "(e='%s', d='%s')" % (e, d)
#if e:
#msg += e
#else:
#if isinstance(d, list): d = d[0]
#(documentId, path) = document.getDocumentVersion(d)
#msg += "version documentId: %s; path: %s\n" % (documentId, path)
msg += 80*'_' + "\n\n"
# AlbumHierarchy checkDisc() and related functions
def checkDisc(self):
"""Determine type of disc and then use appropriate specialized function, e.g. checkAudioCd().
Also gather some information such as volumen label, publisher, publishing date.
This also pre-sets self.maintype which may be re-set later,
important for format columns.
TODO: dvd+rw-mediainfo /dev/dvd tells if it is a DVD-ROM etc.
It can tell if it's prerecorded (DVD-ROM) or DVD-R etc.,
if it's a DVD or CD (takes a while, says ":-( non-DVD media mounted, exiting..."
http://wiki.dvdlookup.org/index.php?title=Disc_Identification (but dvdlookup.org seems to be dead)
lsdvd -x /dev/dvd
http://ivs.cs.uni-magdeburg.de/bs/lehre/wise0001/bs2/iso/docs/iso.html#PVD
"""
log = logging.getLogger("checkDisc")
if not self.setDevice(): return
btc("eject -t " + self.device)
caption = "Checking disc in " + self.device
self.clear()
(self.ean, ok) = QInputDialog.getText(self, caption, "Please enter EAN (or leave blank)", QLineEdit.Normal)
if not ok: return
msg = "Please enter mediaid (or leave blank)"
(self.mediaid, ok) = QInputDialog.getText(self, caption, msg, QLineEdit.Normal)
if not ok: return
headerDic = getHeaderDic(self.device)
cdDiscidList = btc("cd-discid " + self.device).split(" ")
log.info("cdDiscidList=%s" % cdDiscidList)
audioCdText = ""
if len(cdDiscidList)>4:
audioCdText = "It has several tracks so it could be an <b>audio CD</b>.<br>"
(videoDvdLabel, videoDvdText) = testForVideoDvd(self.device)
# dvdbackup -I => DVD
# no DVD but CDNAME => CD-ROM
# cd-discid /dev/dvd | wc -w >=6 => audio CD
label = cutoffZero(headerDic["CDNAME"])
if label: labelText = "It has the volume label: <b>%s</b>.<br>" % label
else: labelText = "It has no volume label.<br>"
msg = "<p>Please tell the <b>type of the disc</b> in the drive %s.<br>" % self.device
msg += labelText + audioCdText + videoDvdText
msg += "</p>"
msgBox = QMessageBox()
msgBox.setText(msg)
msgBox.setWindowTitle(caption)
audioCdButton = msgBox.addButton("Audio CD", QMessageBox.ActionRole)
videoDvdButton = msgBox.addButton("Video DVD", QMessageBox.ActionRole)
cdRomButton = msgBox.addButton("CD-ROM", QMessageBox.ActionRole)
uncdRomButton = msgBox.addButton("UnCD-ROM", QMessageBox.ActionRole)
dvdRomButton = msgBox.addButton("DVD-ROM", QMessageBox.ActionRole)
abortButton = msgBox.addButton(QMessageBox.Abort)
if videoDvdLabel: msgBox.setDefaultButton(videoDvdButton)
elif audioCdText: msgBox.setDefaultButton(audioCdButton)
else: msgBox.setDefaultButton(cdRomButton)
msgBox.exec_()
if msgBox.clickedButton() == abortButton: return
if msgBox.clickedButton() == audioCdButton:
self.checkAudioCd()
elif msgBox.clickedButton() == videoDvdButton:
self.checkVideoDvd()
elif msgBox.clickedButton() == cdRomButton:
self.checkCdRom(label, "CD")
elif msgBox.clickedButton() == uncdRomButton:
self.checkCdRom(label, "UnCD-ROM")
elif msgBox.clickedButton() == dvdRomButton:
self.checkCdRom(label, "DVD")
# elif msgBox.clickedButton() == cdRomButton or msgBox.clickedButton() == dvdRomButton:
#elif msgBox.clickedButton() in (cdRomButton, uncdRomButton, dvdRomButton):
#self.checkCdRom(label)
def readFreeDb(self):
"""Return tuple (tracks, albumTitle, trackTitleList) or None.
Album may be unknown, in this case generic titles are returned.
"""
log = logging.getLogger("readFreeDb")
if not self.setDevice(): return None
caption = "Read Audio CD"
archivePath, activePath, activeMedium, mediumPattern = archive.getArchiveSettings()
# Open CD.
try:
cdrom = DiscID.open(self.device)
disc_id = DiscID.disc_id(cdrom)
except Exception as text:
msg = "Error opening or reading from device %s:\n%s" % (self.device, text)
log.error(msg)
errmsg(msg, caption)
return None
tracks = disc_id[1]
log.info("disc_id: %s" % disc_id)
#x = disc_id[0]
self.discid = "%x" % disc_id[0]
cdtitle = "CD"
offlineSetting = settings.getSetting("misc/offline").lower()
offline = (offlineSetting and offlineSetting[0] == 'y')
if offline:
log.warning("Offline mode, no Internet connection")
msgbox("Offline mode, no Internet connection", caption)
query_status = 200 # No error, just offline.
else:
try:
print(disc_id)
(query_status, query_info) = CDDB.query(disc_id)
except IOError:
query_status = None; error = "IOError, maybe no network connection"
# query_info contains unique pair disc_id, category in case of success
# query_status will be one of the following:
# * 200: Success
# * 211: Multiple inexact matches were found
# * 210: Multiple exact matches were found
# * 202: No match found
# * 403: Error; database entry is corrupt
# * 409: Error; no handshake. (client-side error?)
if query_status == 202: error = "No match found"
elif query_status == 403: error = "Database entry is corrupt"
elif query_status == 409: error = "No handshake (client-side error?)"
elif query_status in (200,210,211): error = None
else: error = "Unknown error"
if error:
msg = "CDDB query failed: %s\n" % error
log.error(msg)
errmsg(msg, caption)
if error or offline:
albumTitle = "(Unknown album)"
trackTitleList = []
for i in range(tracks):
trackTitleList.append("track%02d.cdda" % (i+1))
return (tracks, albumTitle, trackTitleList)
if query_status != 200: # Multiple matches, let user select.
# info['category']: The category the audio CD belongs in
# info['disc_id']: The CDDB checksum disc ID of the given disc
# info['title']: The title of the audio CD
stringList = ["%d: %s (%s)" % (i+1, e["title"], e["category"]) for (i,e) in enumerate(query_info)]
s = StringSelector(stringList)
if not s.exec_(): return
res = s.selection()
if not res: return (tracks, '', ["track%02d.cdda" % (i0+1)for i0 in range(tracks)])
i = int(getId(res[0]))-1 # i is the zero-based index from user's selection
query_info = query_info[i]
#cdtitle = query_info["title"]
# Now we have a query_info to read the detail info with album and track titles etc.
cdtitle = query_info['title']
# print "query_info",query_info
(read_status, info) = CDDB.read(query_info['category'], query_info['disc_id'])
# If successful, info contains album and track titles etc.
if read_status==210: error = None
elif read_status==401: error = "Specified entry not found"
elif read_status==402: error = "Server error"
elif read_status==403: error = "Database entry is corrupt"
elif read_status==409: error = "No handshake"
elif read_status==417: error = "Access limit exceeded"
else: error = "Unknown error"
if error:
msg = "No track info due to error: " + error
log.error(msg)
errmsg(msg, caption)
albumTitle = "(Unknown album)"
trackTitleList = []
for i in range(tracks):
trackTitleList.append("track%02d.cdda" % i+1)
return (tracks, albumTitle, trackTitleList)
# Now we have info with album and track titles etc.
self.discid = info['DISCID']
self.maintype = "audio"
charset = 'latin1'
albumTitle = info['DTITLE'].decode(charset)
extTitleList = []
for i0 in range(tracks):
title = info['TTITLE' + repr(i0)].decode(charset).replace('_',' ')
extTitleList.append(title)
return (tracks, albumTitle, extTitleList)
def checkAudioCd(self):
"""Read CDDB (FreeDB) info about audio CD in drive."""
ean, mediaid = self.ean, self.mediaid # TODO: better concept
self.clear()
self.ean, self.mediaid = ean, mediaid
self.maintype = "audio"
rt = self.maintype.title()
res = self.readFreeDb()
if not res: return
(tracks, albumTitle, origDocTitleList) = res
print ds(locals(), "tracks, albumTitle, origDocTitleList")
self.ot = "CD" # TODO: Try to read type (CD, CD-R, CD-RW...)
# rt is product.rt, i.e. "Audio", "Video", "Application" etc., from self.maintype.title()
# albumTitle is title, usually indentical with product.name and document.title
# origDocTitleList is, for albums with several documents, a Python list of document titles, e.g. track titles
# volumelabel is volume label of DVD or CD-ROM
# self.showDiscMetadata(rt, albumTitle, origDocTitleList, mt="113")
isrcDic = {}
# ean = self.ean
# TODO: Check when ISRC is useful and call icedax conditionally.
#res = bt("icedax -J -H -D /dev/dvd 2>&1") # provides ISRCs
#print "res:", res
#print "res.split('\n')", res.split('\n')
#isrcList = [e.split('\r')[2] for e in res.split('\n') if len(e)>40 and e.startswith('\rscanning for ISRCs')]
#print "isrcList ", isrcList
#try:
#isrcDic = dict([(int(e[3:5]),e[11:].strip().replace('-','')) for e in isrcList])
#except ValueError:
#isrcDic = None # no ISRC data
##if len(e)>20 and e[:2]=='T:'])
#print " ! ! ! ! ! !"
#print "isrcDic ", isrcDic
#d = textToDic(res)
#print d
#ean = d["Media catalog number"] if "Media catalog number" in d else None
# self.showDiscMetadata(rt, albumTitle, origDocTitleList, ean=ean, isrcDic=isrcDic)
self.showDiscMetadata(rt, albumTitle, origDocTitleList, ean=ean, mt="113", isrcDic=isrcDic)
def addVideoDvd(self):
"""Add generic video DVD.
"""
volumelabel = None
UnDVD = False
warnings = ''
title = "(Unknown DVD)"
self.maintype = "video"
albumTitle = title
rt = self.maintype.title()
origDocTitleList = []
self.ot = "DVD"
self.showDiscMetadata(rt, albumTitle, origDocTitleList, volumelabel=volumelabel, mt="122")
def checkVideoDvd(self):
"""Analyze DVD.
TODO: dvd+rw-mediainfo /dev/dvd tells if it is a DVD-ROM etc.
It can tell if it's prerecorded (DVD-ROM) or DVD-R etc.,
if it's a DVD or CD (takes a while, says ":-( non-DVD media mounted, exiting..."
http://wiki.dvdlookup.org/index.php?title=Disc_Identification
lsdvd -x /dev/dvd
"""
if not self.setDevice(): return
discid = self.getDiscId()
self.discid = discid
device = self.device
metainfo = btc("dvdbackup -I -i " + device + " 2>&1")
# Note: dvdbackup output of volumelabel has changed
# from original "WORD_WORD" to "Word Word", that is,
# dvdbackup does not show the real, original volumelabel.
# Dvdbackup and other tools more or less try to guess
# the DVD title from the volumelabel, but that can't work
# because the volumelabel is subject to certain
# restrictions (length, character set). Here is dvdbackup output:
# dvdbackup -I -i /dev/sr2 2>&1 | grep "DVD with title"
# Natm 2 De Dvd3
# HandBrakeCLI and blkid seem to only replace underscores with spaces:
# HandBrakeCLI -i /dev/sr2 -t 0 2>&1 | grep "DVD Title:"
# NATM 2 DE DVD3
# blkid /dev/sr2
# /dev/sr2: LABEL="NATM 2 DE DVD3" TYPE="udf"
# lsdvd and volname perform no conversion:
# lsdvd /dev/sr2 | grep "Disc Title"
# NATM_2_DE_DVD3
# volname /dev/sr2
# NATM_2_DE_DVD3
# If you want to be sure to get only the raw volumelabel, use dd:
# dd if=/dev/sr2 bs=1 skip=32808 count=32
# NATM_2_DE_DVD3
volumelabel = unicode(btc("dd if=/dev/sr2 bs=1 skip=32808 count=32")).strip()
# Check for UnDVD
self.dvdSize = getDvdSize(device)
UnDVD = False
if self.dvdSize>9123456789: # TODO: Exact max. DVD capacity?
msgbox("Size exceeds physical DVD capacity, maybe UnDVD!")
UnDVD = True
warnings = ''
for l in metainfo.splitlines(True):
if re.match("BUP and IFO",l):
warnings += l
if warnings!='':
msgbox(warnings + "\n\nProbably UnDVD")
UnDVD = True
if UnDVD:
title = "(Unknown UnDVD)"
else:
# get title proposal
if volumelabel and volumelabel != "DVDVOLUME":
title = volumelabel.replace('_',' ').title().strip()
# msgbox("Title proposal:\n" + title)
else:
title = "(Unknown DVD)"
volumelabel = None
self.maintype = "video"
albumTitle = title
# self.albumItem = self.regItem(None, "album", albumTitle)
# self.albumItem.setFlags(Qt.ItemIsSelectable | Qt.ItemIsEnabled | Qt.ItemIsEditable)
# self.albumItem.setExpanded(True)
# self.setupFormatColumns([])
# item = self.regItem(self.albumItem, "document", title, cnt=1)
# item.setFlags(Qt.ItemIsSelectable | Qt.ItemIsEnabled | Qt.ItemIsEditable)
# for i in range(len(self.baseColumnWidths)): self.setColumnWidth(i,self.baseColumnWidths[i])
rt = self.maintype.title()
origDocTitleList = []
self.ot = "UnDVD" if UnDVD else "DVD"
# rt is product.rt, i.e. "Audio", "Video", "Application" etc., from self.maintype.title()
# albumTitle is title, usually indentical with product.name and document.title
# origDocTitleList is, for albums with several documents, a Python list of document titles, e.g. track titles
# volumelabel is volume label of DVD or CD-ROM
self.showDiscMetadata(rt, albumTitle, origDocTitleList, volumelabel=volumelabel, mt="122")
def prepareAlbum(self):
"""Prepare album directory before archiving.
For example, provide MP3 files with id3v2 tags.
Or provide a DVD directory with a readme file containing metadata
like summary, actors list etc.
TODO:
in data: actors, composers etc.
format data as HTML here?
get product image:
sqlBase = "SELECT id1 FROM rel WHERE ent2=23 AND id2=%s AND relid=" % id
frontImages = db.valueList(sqlBase + "63814") or db.valueList(sqlBase + "53225") # Python list of document IDs
imagepath = document.getAvailableFile(','.join(frontImages)) if frontImages else None
How to prepare opening:
- take background image
- producer logo or
- DVD cover, or
- licence (shareware)
- put data on it:
- title
- year
- producer
- licence text
- reformat it to suit main video
"""
albumItem = self.getAlbumItem()
if not albumItem:
errmsg("Please select album")
return
objectId = getText(albumItem, self.idColumn)
productId = db.readSql("SELECT productId FROM object WHERE id=%s" % objectId, 0, 1)
if not productId:
errmsg("Please select album with product")
return
origDocId = self.getDocId(albumItem)
p = db.idDic("product", productId)
title = p["name"]
data = {"ean":p["ean"], "title":title}
if albumItem:
documentId = self.getDocId(albumItem)
print("documentId", documentId)
if documentId:
doc = db.idDic("document", documentId)
print(doc)
data["review"] = doc["review"]
data["actors"] = document.getArtists(documentId, "3")
try:
import preparedvd
except:
from . import preparedvd_simple as preparedvd
# prepare(dirpath, title, imagepath, dic):
dir = self.dirToImport or archive.selectDir() # TODO: replace self.copydir
if not dir: return
imagePath = product.getProductImageFile(productId)
if dir: preparedvd.prepare(dir, title, imagePath, data)
def checkCdRom(self, label, ot):
"""Analyze CD-ROM (game etc.).
"""
log = logging.getLogger("checkCdRom")
log.debug("label=%s" % label)
if not self.setDevice(): return
discid = self.getDiscId()
log.info("discid=%s" % discid)
self.discid = discid
title = label.title() if label else '' # TODO: replace '_' with ' '
self.maintype = "application"
rt = self.maintype.title()
origDocTitleList = [title] # TODO: This is th root doc
self.ot = ot #"CD"
self.showDiscMetadata(rt, title, origDocTitleList, mt="111")
def getDiscId(self):
"""Return calculated discid of a disc with filesystem.
Problem: User can mount only what is listed in /etc/fstab.
TODO: correct setup in fstab and settings
TODO: check algorithm, lots of DVDs and CDs have this discid:
d41d8cd98f00b204e9800998ecf8427e
d41d8cd98f00b204e9800998ecf8427e
"""
if not self.setDevice(): return
cmd = "mount %s" % self.device
# print cmd
res = btc(cmd)
# see getMountPoint(device)
realDevicePath = os.path.realpath(self.device)
cmd = "cat /etc/mtab | egrep '^%s' | cut -d' ' -f2" % realDevicePath
# print(cmd)
path = btec(cmd)
# print("getDiscId: path=", path)
res = getDirDiscId(path)
return res
cmd = "umount '%s'" % path
# print cmd
btec(cmd) # TODO: 1. more pythonic; 2. handle errors, e.g. if unmount is not possible
return res
def createIsoFile(self):
"""To copy a CD-ROM, create an ISO file with dd.
TODO: Check those options and if they really work in every case.
I had a case where the directory entries were there but the files were unreadable.
TODO: Check if it also (and better?) works with mkisofs:
mkisofs -r -o /path_to.iso /dev/cdrom
# TODO: Create filelist entry at the same time.
"""
if not self.setDevice(): return
caption = "createIsoFile"
msg = "Enter title"
#device = "/dev/cdrom"
# Tip with isoinfo came from http://www.troubleshooters.com/linux/coasterless.htm
# isoinfoText = btc("isoinfo -d -i " + self.device)
isoinfoText = btc("isoinfo -d -i " + self.device)
isodata = readDicFromText(isoinfoText)
title = readVal(isodata, 'Volume id') or readVal(isodata, 'System id') or ''
# title = ''
# if 'Volume id' in isodata: title = isodata['Volume id']
# elif 'System id' in isodata: title = isodata['System id']
title = title.title()
(title, ok) = QInputDialog.getText(None, caption, msg, QLineEdit.Normal, title)
if not ok: return
volsize = readVal(isodata, 'Volume size is')
blocksize = readVal(isodata, 'Logical block size is')
options = "bs=%s count=%s" % (blocksize, volsize) if volsize and blocksize else 'bs=2048 conv=notrunc'
archivePath, activePath, activeMedium, mediumPattern = archive.getArchiveSettings()
path = uniqueFileName(archivePath + "toarchive/" + archive.createFilename(unicode(title)) + ".iso")
# add " conv=notrunc,noerror"?
cmd = "dd if=%s of='%s' %s" % (self.device, path, options)
# print(cmd)
res = btec(cmd)
def getAlbumItem(self):
"""Return album item (user selects if there are several), or None.
"""
albums = []; strings = []; i = 1
for item in traverseRecursive(self.invisibleRootItem()):
if getText(item, self.levelColumn) == "album":
albums.append(item)
strings.append("%d: %s" % (i, getText(item, self.dbTitleColumn)))
i += 1
if len(albums) == 1: return albums[0]
s = StringSelector(strings)
if not s.exec_(): return
index = int(getId(s.selection()[0])) - 1
# print "index:", index
return albums[index]
def getCurrentAlbumItem(self):
"""Return current album item, or None.
"""
albums = []; strings = []; i = 1
for item in traverseRecursive(self.invisibleRootItem()):
if getText(item, self.levelColumn) == "album" and getText(item, self.curColumn) == ">":
albums.append(item)
if len(albums) == 1: return albums[0]
errmsg("More than one album marked as current!", "AlbumForm")
# def createAlbumCopy(self):
# """Create audio CD-R[W] from audio CD, DVD+-R[W] from video DVD, etc.
# TODO: Currently works for audio CDs from MP3 files only.
# 1. Find album (object) with getText(item, self.curColumn) == ">"
# 2. Call albumCopyCreater(objectId)
# albumCopyCreater does:
# 1. Decide which source(s) to use.
# There are sources for the whole album (e.g. DVD ISO file, MP3WRAP)
# or single documents (e.g. tracks).
# Sources may be more (e.g. MP3) or less easily reconvertible to semi-digital (WAV) format.
# 2. Make sources available.
# 3. Convert sources if necessary.
# 4. Combine them to a CD/DVD image.
# 5. Burn the image (one or more times).
# """
# caption = "createAlbumCopy"
## albumItem = self.getAlbumItem()
## if not albumItem: return
# h = self.hierarchyWidget
## msgbox("self.objectId" + self.objectId)
## object.reproduceAlbum(self.objectId)
# tools.reproduceAlbum(self.objectId)
# return
# documentidList = []
# for item in traverseRecursive(self.invisibleRootItem()):
# if unicode(item.text(self.levelColumn)) == "document": # and getText(item, self.curColumn) == ">":
# documentidList.append(getText(item, self.idColumn))
# # see http://www.faqs.org/docs/Linux-mini/MP3-CD-Burning.html
# print documentidList
# basedir = "/unibas_archive/tmp"
# outdir = basedir + "/cd/"
# btc("mkdir " + outdir)
# for (i0, documentId) in enumerate(documentidList):
# (versionId, mp3file) = document.getDocumentVersion(documentId)
# trackpath = "%strack%02d.wav" % (outdir, i0+1)
# # --rate 44100 --stereo --buffer 3072 --resync
# cmd = "lame --decode '%s' '%s'" % (mp3file, trackpath)
# print cmd
# btc(cmd)
#
# msgbox(outdir + " finished.")
#
# burnCmd = 'wodim -v -pad gracetime=2 dev=/dev/scd0 -audio '
# fl = getFileList(outdir); fl.sort(); trackCount = len(fl)
# for file in fl:
# burnCmd += '"%s/%s" ' % (outdir,file)
# # Output example (to stdout):
# # Track 01: 7 of 87 MB written (fifo 100%) [buf 99%] 10.6x.
# print "burnCmd:" , burnCmd
# pp = ProcessProgress(burnCmd, windowTitle=caption,
# labelText="Burning CD",
# outRegExpString=".*Track\\s*(\\d+):\\s*(\\d+)\\s*of\\s*(\\d+)\\s*MB.*")
# pp.extractExpr = "100*((float(m.group(1))-1)+float(m.group(2))/float(m.group(3))) / %d" % trackCount
# pp.start()
# if QMessageBox.question(None, caption, "Delete %s?" % outdir,
# QMessageBox.StandardButtons(QMessageBox.Yes | QMessageBox.No)) \
# == QMessageBox.No: return
# btc('rm -rf "%s"' % outdir)
def createMp3wrap(self):
"""
"""
caption = "createMp3wrap"
#albumItem = self.getCurrentAlbumItem()
albumItem = self.currentItem()
if not albumItem: return
if getText(albumItem, self.levelColumn) != "album":
errmsg("Please select an album item!", caption)
return
#h = self.hierarchyWidget
albumTitle = getText(albumItem, self.dbTitleColumn)
msgbox(albumTitle)
documentidList = []
# for item in traverseRecursive(self.invisibleRootItem()):
for item in traverseRecursive(albumItem):
# if getText(item, self.levelColumn) == "document" and getText(item, self.curColumn) == ">":
if getText(item, self.levelColumn) == "document":
documentidList.append(getText(item, self.idColumn))
# see http://www.faqs.org/docs/Linux-mini/MP3-CD-Burning.html
# print(documentidList)
dest = uniqueFileName("/unibas_archive/export/" + archive.createFilename(albumTitle))
cmd = "mp3wrap '%s' " % dest
for (i0, documentId) in enumerate(documentidList):
(versionId, mp3file) = document.getDocumentVersion(documentId)
cmd += "'%s' " % mp3file
# print(cmd)
btc(cmd)
fulldest = dest + "_MP3WRAP.mp3"
if os.path.isfile(fulldest):
msgbox("successfully created: " + fulldest, caption)
else:
errmsg("Failed to create: " + fulldest, caption)
# AlbumHierarchy fillFields() and its helper functions
def fillFields(self, objectId=None, productId=None, seriesId=None, docsToA=None):
"""Display data from given album (or package) in hierarchyWidget.
May be called:
- "empty", i.e. without any disc or directory to archive
- when archiving a CD/DVD
- when importing a directory
Can be called with:
- series
- package without object (on package level)
- album
One of these must be given.
TODO: if medium product is known, but its record has no discid yet,
add discid; example: Asterix in America
"""
self.invisibleRootItem().takeChildren()
self.setHeaderLabels(self.baseColumnNames)
if objectId: self.objectId = objectId
if productId and not objectId:
(objectId, descr) = dbq.getOrigObjectOfProduct(productId, objectId)
mainOt = otHead(self.ot or '')
log = logging.getLogger("fillFields")
series = Series(objectId, productId, seriesId, mainOt)
seriesItem = self.regItem(None, "series", series.getName(), id=seriesId)
if not docsToA: docsToA = DocsToArchive()
self.docsToA = docsToA
for se in series.getElements(): # Loop over episodes.
# log.debug("cnt=%s, objectId=%s, productId=%s" % (eCnt, eObjectId, eProductId))
package = Package(se.objectId, se.productId)
packageItem = self.regItem(seriesItem, "package", package.getName(),
id=package.getId(), rn=se.cnt)
for pe in package.getElements():
album = pe.getObjectDic()
if not album: continue
objectId = album["id"]
# if pe.getMaintype(): self.maintype = pe.getMaintype()
self.maintype = pe.getMaintype() or self.maintype
origDocs = docsToA.getOrig(pe.cnt)
copyDocs = docsToA.getCopy(pe.cnt)
cur = '*' if origDocs else '>' if objectId == self.objectId else ' '
albumItem = self.regItem(packageItem, "album", album["descr"], cur=cur,
id=objectId, rn=pe.cnt if package.getId() else se.cnt)
self.showMediaCount(pe.getProduct().getId(), albumItem)
self.setupFormatColumns(package.getVersionMimetypeIds(pe.cnt))
self.displayDocumentsOfObject(objectId, albumItem,
origDocs, copyDocs, album, cur)
self.setColWidths() # also sets format columns
if self.invisibleRootItem().childCount(): # to display a document if there is...
self.setCurrentItem(self.invisibleRootItem().child(0))
# Propose to schedule and execute copy actions.
if self.discid and objectId: # albumList[0]:
newCopies = self.scheduleDefaultActions(docsToA)
if newCopies > 0:
if askYesNo(True, "Execute scheduled actions?"):
self.form.execScheduledActions()
def regItem(self, parent, level, dbTitle=None, file=None, cur=' ',
rn=None, dn=None, id=None, extTitle=None, rnFormat="%02d"):
"""Register and return item of the hierarchy (package, album etc.).
If id is None just return parent.
Arguments:
parent: None if this item is top level, parent item otherwise
level: text to show in level column
title: text to show in title column
rn: running number per parent level, integer counter value
dn: document number per album object, max 2 digits
id: id of original document (not product); if None just return parent
extTitle: title from FreeDB or other external infosource
"""
if not id: return parent
item = QTreeWidgetItem(parent)
if not parent: self.insertTopLevelItem(0, item)
item.setFlags(Qt.ItemIsSelectable | Qt.ItemIsEnabled | Qt.ItemIsEditable)
item.setText(self.levelColumn, level)
item.setExpanded(True)
if level == "album":
item.setChildIndicatorPolicy(QTreeWidgetItem.ShowIndicator) # ChildIndicatorPolicy
item.setExpanded(cur=='>')
if dbTitle: item.setText(self.dbTitleColumn, dbTitle)
if extTitle: item.setText(self.extTitleColumn, extTitle)
if file: item.setText(self.fileColumn, file)
item.setText(self.curColumn, cur)
if rn: item.setText(self.rnColumn, rnFormat % rn)
if dn: item.setText(self.dnColumn, "%02d" % dn)
if id: item.setText(self.idColumn, id)
color = self.backgroundColor[level]
if color:
for c in range(self.fileColumn + 1):
item.setBackgroundColor(c, QColor(color))
return item
def showMediaCount(self, productId, album):
"""Show how many own media of this product the user has."""
if productId:
product = getEntityDic("product", productId)
self.baseColumnNames[self.albumColumn] = product["ot"] or "Album"
self.setHeaderLabels(self.baseColumnNames)
ownMediaCount = db.readSql("SELECT count(*) FROM object WHERE productid=%s AND roomid IS NOT NULL" % productId)
if int(ownMediaCount) > 0: album.setText(self.albumColumn, ownMediaCount)
def setupFormatColumnsFromFiles(self, dir, filelist):
self.setupFormatColumns(self.getFormatIds(dir, filelist))
def setupFormatColumns(self, newids):
"""Add newids to the set of formats and setup format columns.
Create the new formats and columns incl. captions.
Argument: list of mimetype IDs
"""
log = logging.getLogger("setupFormatColumns")
log.debug("self.formats: %s" % unicode([f.caption() for f in self.formats]))
log.info("baseColumnNames: %s" % unicode(self.baseColumnNames))
ids = set([f.getId() for f in self.formats]) # Existing mimetype IDs
log.debug("ids: %s" % unicode(ids))
log.debug("newids: %s" % unicode(ids))
nextCol = self.formatColumnStart + len(ids) # Start column for new IDs
for id in newids: # Loop through IDs some of which may have a column already
# format = mimetype.mimetypeFromFile(dir+'/'+path)
# id = format.getId()
if id not in ids: # Really new, not yet in ids
log.info("New format id: %s" % id)
# print "new id:", id
ids.add(id)
format = mimetype.Mimetype(id)
format.readFromDb()
# self.formatColumn[id] = nextCol; nextCol += 1
self.formatColumn[id] = nextCol; nextCol += 1
self.formats.append(format)
self.columnNames = []; self.columnNames.extend(self.baseColumnNames)
# self.columnNames = self.baseColumnNames # WARNING: this fails, they are pointers to same object
for format in self.formats: self.columnNames.append(format.caption())
log.info("columnNames: %s" % unicode(self.columnNames))
self.columnCount = self.baseColumnCount + len(self.formats)
self.setColumnCount(self.columnCount)
self.setHeaderLabels(self.columnNames)
for i in range(len(self.formats)):
self.setColumnWidth(self.formatColumnStart + i, 30)
def addFormatColumn(self, id):
"""Add a single format column given by ID."""
#h = self #.hierarchyWidget
ids = set([f.getId() for f in self.formats])
if id in ids: return
nextCol = self.formatColumnStart + len(ids)
format = mimetype.Mimetype(id)
ids.add(id)
format.readFromDb()
self.formatColumn[id] = nextCol
self.formats.append(format)
self.columnNames = []; self.columnNames.extend(self.baseColumnNames)
for format in self.formats: self.columnNames.append(format.caption())
self.columnCount = self.baseColumnCount + len(self.formats)
self.setColumnCount(self.columnCount)
#print "self.columnNames",self.columnNames
self.setHeaderLabels(self.columnNames)
for i in range(len(self.formats)):
self.setColumnWidth(self.formatColumnStart + i, 30)
def displayRootCopy(self, dbdocs, album, objectId):
"""If there is a document for the album as a whole,
display its number of copies per type."""
for d in dbdocs:
if d["file"] != '/': continue
album.setText(self.fileColumn, '/')
# As album object ID is needed, it is stored in ID field, not the root document ID.
# The root document ID must be read from DB if needed.
fSql = "SELECT distinct(copytypeid), count(*) FROM v_albumversion WHERE albumid=" + objectId
fSql += " AND origid=%s GROUP BY copytypeid;" % d["documentid"]
for f in db.dicList(fSql):
album.setText(self.formatColumn[f["copytypeid"]], f["count"])
return
def displayDocumentsOfObject(self, objectId, albumItem, origDocs, copyDocs, objectDic, cur):
"""Given an albumItem and its objectId,
display its documents, and sides and works if there are.
Used by fillFields().
origDocs: list of original (non-archivable) documents
copyDocs: list of copy documents, e.g. MP3
TODO: Also handle level "work"? How to recognize it?
"""
log = logging.getLogger("displayDocumentsOfObject")
h = self
dbdocs = db.dicList("SELECT * FROM document d, cp WHERE d.id=cp.documentid "
"AND cp.objectid=%s ORDER BY cp.file, cp.id" % objectId)
# dbdocs: dicList of documents read from DB;
# these are level "document" or "album" (if album, file=='/')
self.displayRootCopy(dbdocs, albumItem, objectId)
sided = objectDic['ot'] in ('MC', 'LP')
if sided:
log.info("This album has sides A and B")
oldside = 'X' # Start with no side, to detect beginning of side A.
sides = ['A','B']
log.debug("album: " + unicode(albumItem))
log.debug("dbdocs: " + unicode(dbdocs))
# log.debug("copyDocs: " + unicode(copyDocs))
workCnt = cnt = extidx = 0; oldWorkId = None
albumdocs = [d for d in dbdocs if d["file"] == '/']
# maxCnt = max(len(dbdocs)-len(albumdocs), len(extTitleList)) + len(albumdocs)
maxCnt = max(len(dbdocs) - len(albumdocs), len(copyDocs)) + len(albumdocs)
log.debug("maxCnt: " + unicode(maxCnt))
rn = 0
for i in range(maxCnt): # index of dbdocs, 0-based loop over documents
i1 = i + 1 # 1-based
# dbdoc is current document if there is one for this index
# (index may be higher if extTitleList is longer).
dbdoc = dbdocs[i] if len(dbdocs)>=i+1 else None # formerly d
src = copyDocs[i].fileName() if i < len(copyDocs) and copyDocs[i] else ''
file = dbdoc.get("file") if dbdoc else src
log.info("file: %s" % file)
if file == '/': continue # Root is handled above.
cnt += 1 # Counter per side, 1-based
documentId = dbdoc.get("documentid") if dbdoc else None
dbTitle = dbdoc.get("title") if dbdoc else ''
log.debug("i=%d cnt=%d documentId=%s dbTitle='%s'" % (i,cnt,documentId,dbTitle))
if sided:
side = file[0] if file else '-'
if side not in sides: side = 'A'
print "side:", side
if oldside != side: # Detect side change, also beginning of first side.
rn = 0 # Reset counter per side.
# RN as Running Number, counts elements of next-higher hierarchy
# DN as Document Number, counts documents of album
# different only if there are sides A/B or works
# TODO: make all fields read-only;
# edit name field only on context menu request
sideItem = self.regItem(albumItem, "side", side, cur=cur, id=side)
oldside = side
parent = sideItem
else:
parent = albumItem
# Check if this document is part of a container (e.g. of a composition)
containerList = document.getContainerDocuments(documentId) if documentId else None
workId = containerList[0] if containerList else None
if workId:
if workId != oldWorkId:
#rn = 0
#oldWorkId = workId
#workCnt += 1
rn, oldWorkId, workCnt = 0, workId, workCnt + 1
workTitle = db.descr("document", workId)
workItem = self.regItem(parent, "work", workTitle, rn=workCnt, id=workId, cur=cur)
parent = workItem
rn += 1
# RN is workCnt if there is a work, else counter per side if sided
# DN is document counter per object
#rn = workCnt if workId else None
item = self.regItem(parent, 'document', dbTitle, file=file, rn=rn, dn=cnt, id=documentId, cur=cur)
if extidx < len(copyDocs): # Display ext. title if there is for this index.
#item.setText(self.extTitleColumn, docList[extidx][0].vd["title"])
item.setText(self.extTitleColumn, copyDocs[extidx].vd["title"])
extidx += 1 # Counter for extTitleList, 0-based
if documentId:
fSql = "SELECT distinct(copytypeid), count(*) FROM v_albumversion WHERE albumid=" + objectId
fSql += " AND origid=%s GROUP BY copytypeid;" % documentId
for f in db.dicList(fSql):
item.setText(self.formatColumn[f["copytypeid"]], f["count"])
#def displaySingleDocuments(self, albumItem, dbdocs, origDocs, copyDocs, object, objectId, cur):
#"""Display documents in an albumItem (lowest level).
#dbdocs: dicList of documents read from DB;
#these are level "document" or "album" (if album, file=='/')
#docList: list of (orig,copy) Document tuples for tracks etc. on disc
#TODO: Also handle level "work"? How to recognize it?
#"""
#log = logging.getLogger("displaySingleDocuments")
#h = self
#print "=== === === === === "
#sided = object['ot'] in ('MC', 'LP')
#print "sided:", sided
#if sided:
#log.info("This album has sides A and B")
#oldside = 'X' # Start with no side, to detect beginning of side A.
#sides = ['A','B']
#log.debug("album: " + unicode(albumItem))
#log.debug("dbdocs: " + unicode(dbdocs))
## log.debug("copyDocs: " + unicode(copyDocs))
#workCnt = cnt = extidx = 0; oldWorkId = None
#albumdocs = [d for d in dbdocs if d["file"]=='/']
## maxCnt = max(len(dbdocs)-len(albumdocs), len(extTitleList)) + len(albumdocs)
#maxCnt = max(len(dbdocs)-len(albumdocs), len(copyDocs)) + len(albumdocs)
#log.debug("maxCnt: " + unicode(maxCnt))
#for i in range(maxCnt): # index of dbdocs, 0-based loop over documents
#i1 = i + 1 # 1-based
## dbdoc is current document if there is one for this index
## (index may be higher if extTitleList is longer).
#dbdoc = dbdocs[i] if len(dbdocs)>=i+1 else None # formerly d
#src = copyDocs[i].fileName() if i < len(copyDocs) and copyDocs[i] else ''
#file = dbdoc.get("file") if dbdoc else src
#log.info("file: %s" % file)
#if file == '/': continue # Root is handled above.
#cnt += 1 # Counter per side, 1-based
#documentId = dbdoc.get("documentid") if dbdoc else None
#dbTitle = dbdoc.get("title") if dbdoc else ''
#log.debug("i=%d cnt=%d documentId=%s dbTitle='%s'" % (i,cnt,documentId,dbTitle))
#print "sided2:", sided
#if sided:
#side = file[0] if file else '-'
#if side not in sides: side = 'A'
#print "side:", side
#if oldside != side: # Detect side change, also beginning of first side.
##cnt = 1 # Reset counter per side.
## TODO: decide if cnt is abs. or rel. per side
## must be abs if used as counter for imported files
## maybe both abs and rel counter?
## Proposal:
## RN as Running Number, counts elements of next-higher hierarchy
## D# or DN as Document Number, counts documents of album
## TODO: make all fields except title read-only;
## TODO: react on title change automatically after ENTER
#sideItem = self.regItem(albumItem, "side", side, cur=cur, id=side)
#oldside = side
#parent = sideItem
#else:
#parent = albumItem
## Check if this document is part of a container (e.g. of a composition)
#containerList = document.getContainerDocuments(documentId) if documentId else None
#workId = containerList[0] if containerList else None
#if workId:
#if workId != oldWorkId:
#oldWorkId = workId
#workCnt += 1
#workTitle = db.descr("document", workId)
#workItem = self.regItem(parent, "work", workTitle, id=workId, cur=cur)
#parent = workItem
#item = self.regItem(parent, 'document', dbTitle, file=file, cnt=cnt, id=documentId, cur=cur)
#if extidx < len(copyDocs): # Display ext. title if there is for this index.
##item.setText(self.extTitleColumn, docList[extidx][0].vd["title"])
#item.setText(self.extTitleColumn, copyDocs[extidx].vd["title"])
#extidx += 1 # Counter for extTitleList, 0-based
#if documentId:
#fSql = "SELECT distinct(copytypeid), count(*) FROM v_albumversion WHERE albumid=" + objectId
#fSql += " AND origid=%s GROUP BY copytypeid;" % documentId
#for f in db.dicList(fSql):
#item.setText(self.formatColumn[f["copytypeid"]], f["count"])
def setColWidths(self):
h = self #.hierarchyWidget
# if not self.maintype:
# self.maintype = (self.productWidget.getValue("rt") or '').lower()
if self.maintype and self.maintype in self.defaultFormats:
for f in self.defaultFormats[self.maintype]:
# print "self.addFormatColumn(%s)" % f
self.addFormatColumn(f)
for i in range(len(self.baseColumnWidths)):
name = self.baseColumnNames[i]
width = self.baseColumnWidths[i]
if name == "External title" and not (self.discid or self.dirToImport):
width = 30
self.setColumnWidth(i, width)
#def getSeriesObjectProductList(self, thisObjectId, thisProductId, thisProduct):
#"""Get a list of tuples (counter, objectId, productId, None)
#if a document on medium thisObjectId is an episode of a document series.
#The tuple members are:
#counter: ord field in the episode rel
#objectId: ID of album carrying an episode
#productId: ID of product of album
#None: placeholder for package ID
#Currently, series containing packages are not yet handled (TODO).
#Albums in packages in a series are shown without package.
#If the current album is not episode of a series, then
#the normal getObjectProductList() function is called which also handles packages.
#"""
#mainOt = otHead(self.ot or '')
#seriesId = dbq.getSeriesIdFromAlbum(thisObjectId)
#if not seriesId: return self.getObjectProductList(thisObjectId, thisProductId, thisProduct)
#self.seriesItem = self.regItem(None, "series", db.descr("document", seriesId), id=seriesId)
#return dbq.getEpisodeObjectList(seriesId, mainOt)
#def getObjectProductList(self, thisObjectId, thisProductId, thisProduct):
#"""Get a list of tuples (counter, objectId, productId, packageId) for non-series episodes.
#Listed are all parts of the package this album is part of.
#If there is no package, the list contains only this album.
#The tuple members are:
#counter: order of package members
#objectId: ID of album in package, or current album ID
#productId: ID of product of album, or None if there is no product
#packageId: ID of package this album is part of, or None
#"""
#log = logging.getLogger("getObjectProductList")
#log.debug("thisObjectId:"+ thisObjectId)
#self.seriesItem = None # Used as parent item.
#prods = db.dicList("SELECT * FROM product WHERE packageid=%s" % thisProductId) if thisProductId else None
#if prods:
#package = getEntityDic("product", thisProductId)
#else:
#package = getEntityDic("product", packageId) # Package details or None.
#if thisProductId: # There is a product.
#log.debug("thisProductId:"+ thisProductId)
#if package: # There is also a package. Produce a list of package members.
#packageId = package["id"]
#log.debug("packageId:"+ packageId)
#(e,l) = db.getPackage(packageId, thisObjectId, thisProductId)
#if e: errmsg(e)
#objectProductList = [] # Now create final list with 1-based counter.
#for (i0, (objectId, productId, z)) in enumerate(l): # z was only needed for sorting.
#objectProductList.append((i0+1, objectId, productId, packageId))
#else: # No package. Return one-element list of this album with its product.
#objectProductList = [(1, thisObjectId, thisProductId, None)]
#else: # No product. Return one-element list of this album without product.
#objectProductList = [(1, thisObjectId, None, None)]
#return objectProductList
def setImportPackage(self):
"""For importDirectory() get package info from self.dirToImport.
Set self.package and self.albumDirList."""
self.package = None
dirIsPackage = False
subdirList = getSubdirList(self.dirToImport) or [] # Sorted subdirectories ...
# ... which are the parts of a package unless they are "AUDIO_TS", "VIDEO_TS" of a video DVD ...
if subdirList: dirIsPackage = not isDvdStructure(self.dirToImport, self.ot) # ... which is checked here.
# log.info("dirIsPackage: %s" % dirIsPackage)
if dirIsPackage: # Now prepare package and albumDirList which contains album directories.
self.albumDirList = [self.dirToImport + '/' + d for d in subdirList] # Sorted list of parts of package.
packageId = self.getDirPackageId(self.dirToImport, subdirList)
if not packageId: return
# packageOt = "%d %ss" % (len(subdirList), ot)
#self.package = db.idDic("product", packageId) # TODO: as Product, see below:
self.package = product.Product(packageId)
self.package.readFromDb()
#packageTitle = package.vd["name"]
#log.info("Selected package ID: %s" % packageId)
askAndRecordProductImage(self.ot, self.dirToImport, packageId)
else: # User-selected directory to archive is not a package but a single album.
self.albumDirList = [self.dirToImport] # Turn single album into one-element list with no package.
def setImportProduct(self, albumDir):
"""For importDirectory() get product number self.prodNo.
Return productId which may be None if user aborts."""
self.otNo = "%s %d" % (self.ot, self.prodNo) # E.g. "CD 1"; ot is parent dir e.g. "MC", "CD", "DVD"
# productDirOnly = albumDir.split('/')[-1] # dirname, in packages this may be just "CD 1", "CD 2" etc.
productDic = readProductDir(albumDir) # Extract product info from dir name.
if "title" in productDic:
print "productDic['title']", productDic['title']
productDic['title'] = productDic['title'].replace('_', ' ')
print "productDic['title']", productDic['title']
productDic["ot"] = self.ot
if self.ot == "DVD":
productDic["rt"] = "Video"
elif self.ot in ("CD","MC","LP"):
productDic["rt"] = "Audio"
elif self.ot == "book":
productDic["rt"] = "text"
if self.ot == "DVD" and productDic["rt"] == "Video":
# Decide if albumDir is a DVD directory structure; in that case, normalize it.
print("albumDir", albumDir)
if os.path.exists(albumDir + "/video_ts") or os.path.exists(albumDir + "/VIDEO_TS"):
print("upcaseDvdDir")
upcaseDvdDir(albumDir)
if not "title" in productDic or productDic["title"] == self.otNo: # If no (good) title is given ...
albumTitle = albumDir.split('/')[-1] or "(Unknown)" # This may be just "CD 1" etc., in this case better...
if self.package: albumTitle = "%s %s" % (self.package.vd["name"], albumTitle) # ... prepend packageTitle
#print "prepend packageTitle =>", albumTitle
productDic["title"] = albumTitle
# log.info("%s %d (1-based) of %d: %s" % (ot, i, len(albumDirList), albumTitle))
if self.package: productDic["packageid"] = self.package.vd["id"]
productId = self.setProduct(productDic, productDic["title"], self.prodNo) # Prepare, ask and evaluate product.
self.productDic = productDic
return productId
def handleImportDvd(self, albumDir):
"""For importDirectory() handle a directory that contains a video DVD."""
log = logging.getLogger("handleImportDvd")
# h = self.hierarchyWidget
rootDocDic = db.getRootDocDic(self.objectId)
# offer user one default and maybe several alternative supported formats
# to select or confirm as copy formats TODO
if rootDocDic: # A root document exists already.
origDoc = document.Document(initDic=rootDocDic, src='/')
# docId = rootDocDic["id"]
# orig.vd = db.idDic("document", docId)
else:
docDic = readProductDir(albumDir, src='/') # Extract doc info from dirname.
origDoc = document.Document(initDic=docDic) # Create (original, non-archivable) document.
#docDic.update(orig.vd)
origDoc.vd["licenseid"] = '13' # DVD is usually copy protected
origDoc.vd["mimetypeid"] = '122' # non-archivable original DVD content
copyDoc = deepcopy(origDoc)
copyDoc.vd["mimetypeid"] = "122" # it is still a directory, only after genisoimage it's an ISO file
copyDoc.src = albumDir
print("in handleImportDvd: copyDoc.src=", copyDoc.src)
# log.info("orig: %s" % unicode(orig))
# log.info("copyDoc: %s" % unicode(copyDoc))
return (origDoc, copyDoc)
def handleImportDoc(self, j, file, fileList, albumDir, albumTitle):
"""Handles a single file in a directory to import; called by importDirectory.
j: 1-based file index
Produces two Documents: oridDoc and copyDoc, mostly identical,
except for mimetype, hash.
"""
# TODO: Show progress in MultiProgressDialog, by file number on lowest level.
# (Not file size because here we only prepare copies.)
#print("file %d of %d" % (j, len(fileList)))
path = albumDir + '/' + file # Complete (source) file path.
schema = "%%s %%%dd" % len(unicode(len(fileList))) # "%s %2d" if there are >=10 files
# log.info("File no. %d: %s" % (j, path))
# hash = archive.hash(path) # Hash of copy to check if it is already known.
#print("* * * Getting hash of "+path)
# TODO: problem: we have hash for file, but not for directory where we move the file
# To save time, we should store the hash and use it if file is moved unchanged.
hash = archive.smarthash(path) # Hash of copy to check if it is already known.
#print("hash:", hash)
dl = db.dicList("SELECT * FROM document WHERE hash='%s'" % hash)
if dl: # Copy already known. TODO: More thoroughly think of what to do in this case.
docDic = dl[0]
origDoc = document.Document(initDic=docDic) # Create (original, non-archivable) document.
msg = "File '%s' is already archived as document:\n%s\n" % (file, db.descrWithId("document", docDic["id"]))
# if not docDic["mimetype"]: msg += "\nBut has no mimetype!"
msg += "Mimetype: %s" % unicode(docDic["mimetypeid"])
# log.warn(msg)
msgbox(msg)
#docDic["copymtid"] = docDic["mimetypeid"]
else:
docDic = readDocFilename(file, schema % (albumTitle, j)) # Extract doc info from filename.
#if singleDoc: docDic["file"] = '/'
origDoc = document.Document(initDic=docDic) # Create (original, non-archivable) document.
attributes = dict() # Will be filled by metadatareader.
metadatareader.readFromFile(attributes, path)
metadatareader.selectMaxLikelihood(attributes)
origDoc.getMetadataValues(attributes)
# print '== == == origDoc.vd["mimetypeid"]', origDoc.vd["mimetypeid"]
copyDoc = deepcopy(origDoc)
# Mimetype is that of copy. Original mimetype (e.g. "SO", "MP") is derived here from it.
if not "mimetypeid" in origDoc.vd: origDoc.vd["mimetypeid"] = None
mimetypeId = copyDoc.vd["mimetypeid"] = origDoc.vd["mimetypeid"]
# Every format, whether supported or not, can be handled in a way
# that a file in this format can be archived, i.e. copied to the archive
# and provided with basic metadata (title etc.).
# So if a mimetype (=format) is recognized, we must ensure that
# there is a format column for it.
if mimetypeId:
#print "mimetypeId", mimetypeId
self.setupFormatColumns([mimetypeId])
mt = mimetype.Mimetype(mimetypeId)
if mt.maintype() == "audio" or self.ot in ("MC","LP"): origDoc.vd["mimetypeid"] = '113'
elif mt.maintype() == "video" and self.ot == "DVD": origDoc.vd["mimetypeid"] = '122'
# log.info("orig mimetypeid: %s; copy mimetypeid: %s" % (docDic["mimetypeid"], mimetypeId))
#print docDic
copyDoc.src = path
# print ">>>>>>>>>> copyDoc:", copyDoc
#docDic["albumDir"] = albumDir
#docDic["file"] = file
return (origDoc, copyDoc)
def importFileAlbum(self):
"""Archive an album stored in a file.
The file represents the product, object and root document.
Create a directory, move file into it and
use importDirectory() for this.
"""
caption = "Import file album"
#h = self.hierarchyWidget
log = logging.getLogger("importFileAlbum")
f = selectFile(self.lastImportedType)
if not f: return
filename = os.path.basename(f)
dirpath = uniqueFileName(os.path.splitext(f)[0])
msg = textwrap.dedent("""
<html><h3>Import File Album</h3>
<p>This function imports single file albums from HD.
It is only a wrapper for the more general ImportDirectory function.
it creates a directory with the single file in it
and calls the ImportDirectory function.</p>
<p>Should this function fail, you will find the file
in this directory:</p>
<blockquote>%s</blockquote>
<p>To try again importing it, use the ImportDirectory function
directly then.</p></html>
""")[1:-1] % dirpath
msgbox(msg, caption)
os.mkdir(dirpath)
newname = "%s/%s" % (dirpath, filename)
os.rename(f, newname)
# If there is a rawfilelist entry for this file, ensure it points
# to the file at the new location.
sql = "UPDATE rawfilelist SET file=%s WHERE file=%s AND host=%s" \
% (q(os.path.realpath(newname)), q(os.path.realpath(f)), q(getHost()))
print sql
db.execSql(sql)
self.importDirectory(dirpath)
def importDirectoryDialog(self):
"""Dialog to input directory to import using importDirectory()."""
dirToImport = selectDir(self.lastImportedType)
if not dirToImport: return
if dirToImport[-1] == '/': dirToImport = dirToImport[:-1]
self.importDirectory(dirToImport)
def importDirectory(self, dirToImport):
"""Archive an album stored in a directory that conforms to specs in node 3401.
Archiving happens in several steps. This is only the first one:
1. (This one): Create DB entries of original docs, remember file paths
This corresponds to showDiscMetadata but they cannot be merged.
Result: product, object and (original) document information in the DB.
At the end, fillFields() is called.
If user agrees, commitImport() is called.
Next see proposals
TODO: Make sure mimetype and license info is stored.
copy contype, license, length, languageid
TODO: DVDs can be imported as ISO files. Don't create directory structure in this case.
"""
caption = "Import directory"
log = logging.getLogger("importDirectory")
self.clear()
self.dirToImport = dirToImport
log.info("Selected directory to import: %s" % dirToImport)
self.ot = ot = parentDir = dirToImport.split('/')[-2] # Parent directory named "CD", "DVD", "MC" or so
self.setImportPackage() # Set self.package and self.albumDirList
log.info("self.package: %s" % unicode(self.package))
log.info("self.albumDirList: %s" % self.albumDirList)
objectId1 = None # Will hold first object ID.
docsToA = DocsToArchive()
# Loop through (possibly 1-element) album list.
for i0, albumDir in enumerate(self.albumDirList):
self.prodNo = i = i0 + 1 # 1-based album counter
# TODO: Make sure that with packages, always the right product is preselected.
productId = self.setImportProduct(albumDir) # sets self.productDic
if not productId: return
askAndRecordProductImage(ot, albumDir, productId) # TODO: think more thoroughly
objectDic = {"ot":ot, "dt":getDimensionType(ot), "virtual":"true",
"descr":albumDir.split('/')[-1], "orig":"true", "productid":productId}
albumTitle = self.productDic["name"]
objectId = self.setObject(objectDic, albumTitle) # Prepare, ask and evaluate object.
if not objectId: return # TODO: in this case stop here and don't create documents
self.objectId = objectId
if not objectId1: objectId1 = objectId # First objectId needed for self.objectId.
#if not self.package: self.objectWidget.setId(objectId)
if isDvdStructure(albumDir):
docsToA.setPair(i, albumList.append([self.handleImportDvd(albumDir)])) # there is only one tuple in the list
else: # Every file is a document
fileList = getFileList(albumDir)
# albumList.append([]) # create albumList[i0]
for (j0, file) in enumerate(fileList): # Loop through files sorted alphabetically.
j = j0 + 1 # 1-based file index
#(origDoc, copyDoc) = self.handleImportDoc(j, file, fileList, albumDir, albumTitle)
docsToA.addPair(i, self.handleImportDoc(j, file, fileList, albumDir, albumTitle))
# albumList[i0].append((orig, copy))
self.objectId = objectId1 # fillFields() needs an objectId to display
self.lastImportedType = ot
# docsToA.pprint()
self.fillFields(objectId=objectId1, docsToA=docsToA)
# To reproduce e.g. a video DVD:
# In AlbumForm, find the album
# File - Reproduce album
# If the copy should get a number, say Yes to "Create new object ib DB?", and
# select the room for this new object
# Note the number and descr of the new object
# Insert the object (maybe after writing number and descr on it)
def reproduceAlbum(self):
tools.reproduceAlbum(self.objectId)
def getDirPackageId(self, dir, subdirList):
"""Return product ID of package. Used in importDirectory.
"""
log = logging.getLogger("getDirPackageId")
package = product.Product(initDic=readProductDir(dir)) # Extract information from directory name.
log.info("package: %s" % unicode(package))
albums = len(subdirList)
ot = parentDir = dir.split('/')[-2] # Parent directory named "CD", "DVD", "MC" etc.
package.vd["ot"] = "%d %ss" % (albums, ot) # Add derived information e.g. ot="2 CDs"
packageId = package.vd["id"] if "id" in package.vd else None
pid = getProductidFromObjectidInDic(package) # If objectid is known, try to get productid from it.
if pid: packageId = pid
if package.vd["title"].endswith(" " + package.vd["ot"]): # Remove "2 CDs" or so at end
package.vd["title"] = package.vd["title"][:-len(package.vd["ot"])-1]
if not packageId:
if not "title" in package.vd:
(text,ok) = QInputDialog.getText(self, "Enter package title",
"Please enter package title here.")
if not ok: return None
title = unicode(text)
else:
title = package.vd["title"]
critList = ["name=%s" % q(title)]
if "ean" in package.vd: critList.append("ean=%s" % q(package.vd["ean"]))
# if self.discid: critList.append("discid=%s" % self.discid)
crit = " OR ".join(critList)
msg = "<html><b>Please select package</b></html>"
f = finddialog.FindEntityAndTypeDialog("product", msg=msg)
f.helpIntro = "<p>Please select or create a package</p>"
f.critEdit.setText(crit)
f.initDic = package.vd
f.newRecordValue = title
f.showCandidates()
packageId = f.selectedId() if f.exec_() else None
return packageId
#def setProduct(self, productDic, albumTitle=None, partno=None): #, origDocTitleList=None):
#"""Prepare, ask for and evaluate user's decision about album product.
#TODO: Ask for barcode first, if given, can be ean, isxn or mediaid, see barcode
#Only return None after asking user to select product via FindDialog.
#TODO: in a package, default-select the right product!
#Arguments:
#productDic: Data gathered so far about product. Used for initDic; name is ignored.
#albumTitle: Title found out so far. Will be prepared and used as name.
#TODO:
#- If discid is known and a product with this discid exists, then
#show this product first (other candidates as well, but this preselected), and
#focus on okButton
#- If product is likely not in DB, set focus on addButton
#TODO: If filename is "NORTHANGER ABBEY.ISO" or "NORTHANGER_ABBEY.ISO"
#take NORTHANGER_ABBEY as product.volumelabel and try to get product from it.
#"""
#a = albumTitle or '(unknown)'
#log = logging.getLogger("setProduct")
##ot = productDic["ot"] if "ot" in productDic else None
##ot = ot or self.ot
#ot = productDic.get("ot") or self.ot
#partnoString = " %d" % partno if partno else ''
#ean = productDic.get("ean")
#productIdListByDiscid = db.idListByFieldFromDic("product", "discid", productDic)
#productId = productDic.get("id")
#if not albumTitle: albumTitle = productDic.get("name")
#if not productId and "objectid" in productDic:
## dl = db.dicList("SELECT productid, descr FROM object WHERE id=" + productDic["objectid"])
#d = db.idDic("object", productDic["objectid"])
#if not d:
## msg = "Album dir %s refers to nonexisting objectid %s" % (albumDir, productDic["objectid"])
#errmsg("No object with id: %s" % productDic["objectid"])
#return None
#productId, albumTitle = d["productid"], d["descr"]
#if not productId and productDic.get("packageid"):
#packageId = productDic["packageid"]
#package = db.idDic("product", packageId)
#if package:
#albumTitle = package["name"]
#if partno:
#parts = getNum(package["ot"]) or 0
#schema = " %s %d" if parts < 10 else " %s %02d"
#albumTitle += schema % (ot, partno) #" %s %d" % (ot, partno)
#crit = "!packageid=%s" % packageId
#msg = "<p><b>Please select %s%s of package:<br>%s<br>that corresponds to this album to import:<br>%s</p></b>" \
#% (ot, partnoString, package["name"], a)
#f = finddialog.FindDialog("product", msg=msg)
## f.order = "levenshtein(name,%s)" % q(a) # TODO: autom. install and activate
#f.newRecordValue = albumTitle
##f.order = "name"
#partnoString = ' ' + unicode(partno) if partno else ''
#f.critEdit.setText(crit)
#f.initDic = productDic
## f.initDic.update({"name":"%s %s%s" % (package["name"], ot, partnoString)})
#f.showCandidates()
## print "productIdListByDiscid", productIdListByDiscid
#if not productIdListByDiscid: #"discid" in productDic and not "ean" in productDic:
## msgbox("f.addNewButton.setFocus()")
#f.addNewButton.setFocus()
#if not f.exec_(): return None
#productId = f.selectedId()
#if not productId:
#critList = []
## crit = " discid=%s" % q(self.discid) if self.discid else " false"
#if self.discid: critList.append("discid=%s" % q(self.discid))
#if 'volumelabel' in productDic:
#critList.append("volumelabel=%s" % q(productDic["volumelabel"]))
#if albumTitle and not albumTitle.startswith("(Unknown"):
#critList.append("name~*%s" % qpr(albumTitle))
#log.debug("critList: %s" % ','.join(critList))
#msg = "Please select %s %d" % (ot, partno) if partno else ''
#f = finddialog.FindDialog("product", msg=msg)
## f.helpIntro = "<p>Please select, confirm or create the product that corresponds to this %s.</p>" % ot
## f.critEdit.setText(crit)
#f.newRecordValue = albumTitle
#f.initDic = productDic
## f.showCandidates()
#f.critList = critList
#f.execCritList()
#productId = f.selectedId() if f.exec_() else None
#if not productId: return None
## From here, we have a productId. Take (maybe new) product name as albumTitle.
#albumTitle = db.idDic("product", productId)["name"]
#self.productId = productId
#if self.discid: db.setValueIfNull("product", "discid", self.discid, productId, True)
#productDic["name"] = albumTitle # needed in importDirectory()
#return productId
def setProduct(self, productDic, albumTitle=None, partno=None): #, origDocTitleList=None):
"""Prepare, ask for and evaluate user's decision about album product.
TODO: Ask for barcode first, if given, can be ean, isxn or mediaid, see barcode
Only return None after asking user to select product via FindDialog.
TODO: in a package, default-select the right product!
Arguments:
productDic: Data gathered so far about product. Used for initDic; name is ignored.
albumTitle: Title found out so far. Will be prepared and used as name.
TODO:
- If discid is known and a product with this discid exists, then
show this product first (other candidates as well, but this preselected), and
focus on okButton
- If product is likely not in DB, set focus on addButton
TODO: If filename is "NORTHANGER ABBEY.ISO" or "NORTHANGER_ABBEY.ISO"
take NORTHANGER_ABBEY as product.volumelabel and try to get product from it.
"""
a = albumTitle or '(unknown)'
log = logging.getLogger("setProduct")
#ot = productDic["ot"] if "ot" in productDic else None
#ot = ot or self.ot
ot = productDic.get("ot") or self.ot
partnoString = " %d" % partno if partno else ''
ean = productDic.get("ean")
print "ean in setProduct():", ean
productIdListByDiscid = db.idListByFieldFromDic("product", "discid", productDic)
productId = productDic.get("id")
if not albumTitle: albumTitle = productDic.get("name")
if not productId and "objectid" in productDic:
# dl = db.dicList("SELECT productid, descr FROM object WHERE id=" + productDic["objectid"])
d = db.idDic("object", productDic["objectid"])
if not d:
# msg = "Album dir %s refers to nonexisting objectid %s" % (albumDir, productDic["objectid"])
errmsg("No object with id: %s" % productDic["objectid"])
return None
productId, albumTitle = d["productid"], d["descr"]
if not productId:
critList = []
# crit = " discid=%s" % q(self.discid) if self.discid else " false"
if self.discid: critList.append("discid=%s" % q(self.discid))
vl = productDic.get('volumelabel')
if vl and len(vl) > 0:
critList.append("volumelabel=%s" % q(productDic["volumelabel"]))
if ean:
crit = "ean=%s" % q(ean)
if len(ean) < 13:
ean13 = '0' * (13 - len(ean)) + ean
crit += " OR ean=%s" % q(ean13)
critList.append(crit)
if productDic.get("packageid"):
critList.append("packageid=" + productDic.get("packageid"))
pkid = productDic.get("packageid")
if pkid: critList.append("packageid=" + pkid)
if albumTitle and not albumTitle.startswith("(Unknown"):
critList.append("name~*%s" % qpr(albumTitle))
log.debug("critList: %s" % ','.join(critList))
msg = "Please select %s %d" % (ot, partno) if partno else ''
f = finddialog.FindDialog("product", msg=msg)
# f.helpIntro = "<p>Please select, confirm or create the product that corresponds to this %s.</p>" % ot
# f.critEdit.setText(crit)
f.newRecordValue = albumTitle
f.initDic = productDic
# f.showCandidates()
f.critList = critList
f.execCritList()
productId = f.selectedId() if f.exec_() else None
if not productId: return None
# From here, we have a productId. Take (maybe new) product name as albumTitle.
albumTitle = db.idDic("product", productId)["name"]
self.productId = productId
if self.discid: db.setValueIfNull("product", "discid", self.discid, productId, True)
productDic["name"] = albumTitle # needed in importDirectory()
return productId
def setObject(self, objectDic, albumTitle):
"""Prepare, ask for and evaluate user's decision about album object.
Only return None after asking user to select product via FindDialog.
Arguments:
objectDic:
albumTitle: Title found out so far. Will be prepared and used as name.
TODO:
If there is object of given product (objectDic["productid"]), then
(pre-)select this (and maybe offer no others?).
Other candidates in order of preference:
object.descr=product.name
object.descr similar to (levenshtein) product.name
Prefer orig=True
Realizable as a list of criteria and orders, not a single one:
[("productid=%s AND orig=True" % productId, ""),
("productid=%s" % productId, ""),
("descr~*%s" % qpr(albumTitle), ""),
]
"""
#h = self.hierarchyWidget
ot = objectDic["ot"]
descr = objectDic["descr"]
if not ot: ot = self.ot
if not "dt" in objectDic: objectDic["dt"] = getDimensionType(ot)
objectDic.update({"ot":ot})
productId = objectDic.get("productid")
critList = []
# if self.objectId: crit += " OR id=" + self.objectId
if self.objectId: critList.append("id=" + self.objectId)
#crit = " productid=%s" % productId if productId else " false"
if productId:
critList.append("productid=%s AND orig=True" % productId)
critList.append("productid=%s" % productId)
if albumTitle and not albumTitle.startswith("(Unknown"):
critList.append("descr~*%s" % qpr(albumTitle))
msg = "<html><b>Please select medium object of<br>%s<br>%s</b></html>" % (albumTitle, descr)
f = finddialog.FindEntityAndTypeDialog("object", msg=msg)
f.critList = critList
# levenshtein is in fuzzystrmatch module, see http://www.postgresql.org/docs/8.3/static/fuzzystrmatch.html
# to activate it, see http://www.postgresql.org/docs/8.3/static/contrib.html
# psql -f /usr/share/postgresql/8.4/contrib/fuzzystrmatch.sql
# f.order = "levenshtein(descr,%s)" % q(descr) # TODO: could be better
f.helpIntro = "<p>Please select, confirm or create the object that corresponds to this %s.</p>" % ot
# f.critEdit.setText(crit)
#msgbox("albumTitle: %s" % albumTitle)
f.newRecordValue = albumTitle
if productId: objectDic.update({"productid":productId})
if not "orig" in objectDic: # New objects are originals by default, because
objectDic.update({"orig":"true"}) # usually one archives originals rather than copies
f.initDic = objectDic
# f.showCandidates()
f.execCritList()
objectId = f.selectedId() if f.exec_() else None
# Problem: If product is selected and later object, but object has no productid,
#it overwrites product with None because of signal.
#Solution: if object exists but has no productid, set productid before setId().
#If it already has a different productid, ask if it should be overwritten.
if objectId:
object = db.dicList("SELECT * FROM object WHERE id="+objectId)[0]
objectProductId = object["productid"]
print("productId, objectProductId:", productId , objectProductId)
#if self.discid: self.productWidget.setValueIfNull("discid", self.discid)
if productId and not objectProductId: db.execSql("UPDATE object SET productid=%s WHERE id=%s" % (productId, objectId))
else: # askAndOverwriteField(entityType, field, oldValue, newValue)
if objectProductId != productId:
msg = "Selected " + db.descrWithEntityAndId("object", objectId)
msg += "\nalready has different product %s." % db.descrWithEntityAndId("product", objectProductId)
msg += "\n\nReplace it with %s?" % db.descrWithEntityAndId("product", productId)
res = QMessageBox.question(self, "Different product", msg,
QMessageBox.StandardButtons(QMessageBox.Cancel | QMessageBox.No | QMessageBox.Yes),
QMessageBox.Yes)
if res == QMessageBox.Cancel: return
if res == QMessageBox.Yes: db.execSql("UPDATE object SET productid=%s WHERE id=%s" % (productId, objectId))
if not object["descr"] or object["descr"] == "(Unknown)":
db.execSql("UPDATE object SET descr=%s WHERE id=%s" % (q(albumTitle), objectId))
self.objectId = objectId
return objectId
def checkForPackagePart(self):
"""Return package product id and the number of the current product in this package,
after asking the user if this product is part of a package.
Return tuple:
package product id, current product number, total products in this package.
"""
log = logging.getLogger("checkForPackagePart")
h = self
# First check if the product (from self.discid) is known to be part of a package.
if self.discid:
#sql = "SELECT id, name, packageid FROM product WHERE discid=%s" % q(self.discid)
#dl = db.dicList(sql)
dl = db.dicList("SELECT id, name, packageid FROM product "
"WHERE discid={}".format(q(self.discid)))
if len(dl) == 1:
productId, packageId = dl[0]["id"], dl[0]["packageid"]
#name = dl[0]["name"]
if packageId:
# TODO: Normally we use object.descr for ordering.
# Here we use product.name which leads to a BUG
# if product.name != object.descr
#sql = "SELECT id FROM product WHERE packageid=%s ORDER BY convert_to(name, 'utf8')" % packageId
#vl = db.valueList(sql)
vl = db.valueList("SELECT id FROM product WHERE packageid={}"
" ORDER BY convert_to(name, 'utf8')".format(packageId))
if productId in vl:
i = vl.index(productId) + 1
packageDescr = db.descr("product", packageId)
# TODO: What if not true?
msg = "Product %s: %s\nseems to be the %d. part in\n%s.\nIs that true?" \
% (productId, dl[0]["name"], i, packageDescr)
# msgbox(msg) # TODO: Ask and return only if confirmed
if askYesNo(True, msg):
return (packageId, i, len(vl))
# In the last 1000 products, find those which are packages (ot='2 CDs' or so)
# and return id, name and number of parts (e.g. CDs)
sql = textwrap.dedent("""
SELECT id, ot, substring(ot from '^[0-9]+ ') as n, name
FROM (SELECT id, ot, name FROM product ORDER BY id DESC LIMIT 1000) AS prod
WHERE ot~'^[0-9]+ %s' ORDER BY id DESC LIMIT 20;
""")[1:-1] % self.ot
log.debug(sql)
lastPackagesDicList = db.dicList(sql)
sqlTrunc = "SELECT COUNT(id) FROM product WHERE packageid="
packList = [] # Gets packages that are not yet full.
for p in lastPackagesDicList: # Loop over all packages, incl. full ones.
existingParts = int(db.readSql(sqlTrunc + p["id"]))
if existingParts < int(p["n"]): # Needs to be filled, so include in selection.
s = "%s #%d of %s from %s: %s %s" \
% (self.ot, existingParts+1, p["n"], p["id"], p["ot"], p["name"])
log.debug(s)
packList.append(s)
if not packList: return (None, 1, 1) # Nothing to select from.
res = StringSelector(packList, "If this is part of a package, please select that package")
if not res.exec_(): return (None, 1, 1) # Nothing selected.
# print "Selected:", s.selection()[0]
if not res.selection(): return (None, 1, 1)
# m = re.compile(".*#([0-9]+) from ([0-9]+):.*").match(s.selection()[0])
regexpString = ".*#([0-9]+) +of +([0-9]+) +from +([0-9]+):.*"
m = re.compile(regexpString).match(res.selection()[0])
if not m:
msg = 'regexpString = %s\n' % regexpString
msg += 'does not match %s\n' % res.selection()[0]
msg += "This shouldn't happen."
print(msg)
errmsg(msg)
return (None, 1, 1) # Shouldn't happen
return (m.group(3), int(m.group(1)), int(m.group(2)))
def showDiscMetadata(self, rt, albumTitle, origDocTitleList,
volumelabel=None, ean=None, mt=None, isrcDic=None):
"""Identify package (if there is), product(s), object(s). Arguments:
rt: product.rt, i.e. "Audio", "Video", "Application" etc.,
from self.maintype.title()
albumTitle: title, usually indentical with product.name and document.title
origDocTitleList: for albums with several documents, a Python list
of document titles, e.g. track titles
volumelabel: volume label of DVD or CD-ROM
"""
if self.ean and not ean: ean = self.ean
log = logging.getLogger("showDiscMetadata")
#log.debug("rt: %s, albumTitle=%s, title list=%s" \
#% (unicode(rt), albumTitle, unicode(origDocTitleList)))
(packageId, partNo, totalParts) = self.checkForPackagePart()
# log.debug("packageId=%s, partNo=%s, totalParts=%s" % (packageId, partNo, totalParts))
log.debug(ds(locals(), "packageId, partNo, totalParts"))
#log.debug("self.maintype=%s", self.maintype)
if packageId: # Use as default for albumTitle
if not albumTitle or albumTitle.startswith("(Unknown"):
package = product.Product(packageId)
package.readFromDb()
albumTitle = "%s %s %d" % (package.vd["name"], self.ot, partNo)
productDic = {"rt":rt or product.rtDefault(ot), "ot":self.ot,
"ean":ean, "discid":self.discid, "volumelabel":volumelabel,
"packageid":packageId, "name":albumTitle}
# log.debug('productDic["rt"]=%s', productDic["rt"])
productId = self.setProduct(productDic, albumTitle, partNo)
#print db.dicList("SELECT id, ot, rt, name FROM product WHERE id=" + productId)[0]
objectDic = {"ot":self.ot, "descr":productDic["name"], "productid":productId}
objectId = self.setObject(objectDic, productDic["name"])
self.objectInDevice = objectId
if self.maintype and self.maintype == "application":
self.correctCdromEntry(albumTitle)
self.emit(SIGNAL("changedId"),objectId)
#docList = [{"title":t} for t in origDocTitleList]
# docList = []
origList, copyList = [], []
for (i0, t) in enumerate(origDocTitleList):
# print "showDiscMetadata", i0
orig = document.Document(initDic={"title":t, "mimetypeid":mt})
if isrcDic and i0+1 in isrcDic: orig.vd["isxn"] = isrcDic[i0+1]
origList.append(orig)
copyList.append(document.Document(initDic={"title":t}))
#log.info("docList: %s" % uc(docList))
# albumList = [docList if i+1==partNo else [] for i in range(totalParts)]
docsToA = DocsToArchive()
#docs.count = totalParts
docsToA.orig = {partNo:origList} # index -> list of Document()s
docsToA.copy = {partNo:copyList} # index -> list of Document()s
self.fillFields(docsToA=docsToA, objectId=objectId)
def correctCdromEntry(self, albumTitle):
"""If a CD-ROM has exactly one copy, with empty or NULL cp.file,
then it is most likely an incorrect, placeholder entry.
Correct it like this:
cp.file='/'
document: title=object.descr, mimetypeid=130, contype='CD-ROM'
"""
h = self #.hierarchyWidget
# title = albumTitle or "(Unknown)"
if not self.objectId: return
cpDicList = db.dicList("SELECT * FROM cp WHERE objectid=" + self.objectId)
if len(cpDicList) != 1: return
cp = cpDicList[0]
if cp["file"]: return # not empty or NULL
db.execSql("UPDATE cp SET file='/' WHERE id=" + cp["id"])
setTitle = "title=%s," % q(albumTitle) if albumTitle else ''
sql = "UPDATE document SET %s mimetypeid=130, contype='CD-ROM', licenseid=10 WHERE id=%s" \
% (setTitle, cp["documentid"])
db.execSql(sql)
def scheduleSingleAction(self, item, formatId, actionChar):
col = self.formatColumn[formatId]
oldtext = (unicode(getText(item, col)) or '') + " "
item.setText(col, oldtext[:2] + actionChar)
def scheduleDefaultActions(self, docsToA):
"""Schedule default actions, such as making copies of audio tracks in MP3 format,
execScheduledActions().
TODO: Under some criteria, schedule a copy of complete CD in one file
instead of one MP3 file per track,
for example for audio books.
- if the single documents are named tracknn.cdda
- if the album is episode of a series
- if it is an audio book
"""
caption = "scheduleDefaultActions"
log = logging.getLogger("scheduleDefaultActions")
#print "self.dirToImport", self.dirToImport
#print "self.formats", self.formats
#msgbox(caption)
if self.dirToImport: return # Do not schedule default actions when called from importDirectory.
if len(self.formats) == 0: return
# self.defaultFormats is a dictionary maintype=>list of formatIds;
# For the first of these formats, a copy per doc is scheduled by default if it does not exist yet.
defaultFormatId = self.defaultFormats[self.maintype][0] # = {"audio":['57','59']}
defaultFormatCol = self.formatColumn[defaultFormatId]
actionChar = 'N'
# TODO: do not ask if copy exists
msgBox = QMessageBox()
msgBox.setText("Copy whole or parts?")
msgBox.setWindowTitle(caption)
rootButton = msgBox.addButton("Whole", QMessageBox.ActionRole)
partsButton = msgBox.addButton("Parts", QMessageBox.ActionRole)
noneButton = msgBox.addButton("None", QMessageBox.ActionRole)
msgBox.setDefaultButton(rootButton if getLevRatio(docsToA)>0.5 else partsButton)
msgBox.exec_()
if msgBox.clickedButton() == noneButton: return
copyLevel = "album" if msgBox.clickedButton() == rootButton else "document"
c = 0 # counter of scheduled copies
for item in traverseRecursive(self.invisibleRootItem()):
cur = getText(item, self.curColumn)
if cur == ' ': continue
level = getText(item, self.levelColumn)
# For video DVDs and CD-ROMS/DVD-ROMs, schedule copy of root, if necessary.
# print 'level="%s"; self.ot="%s"; self.maintype="%s"' % (level, self.ot, self.maintype)
#if level == "document" \
#or (level == "album" and self.maintype == "application") \
#or (level == "album" and self.ot == "DVD" and self.maintype == "video"):
if level == copyLevel:
s = getText(item, defaultFormatCol) or '0'
try:
existingCopies = int(s)
except ValueError:
existingCopies = 0
# Only schedule copy action if there is no copy yet.
if existingCopies == 0:
self.scheduleSingleAction(item, defaultFormatId, actionChar)
c += 1
return c
def ensureDocumentRecordExists(self):
"""Ensure original (non-archivable) document record exists for current item.
Return its documentId is successful, else None."""
item = self.currentItem()
if not item:
errmsg("ensureDocumentRecordExists: No item")
return None
origDocId = self.getDocId(item)
cnt = getText(item, self.rnColumn)
if origDocId:
return origDocId
#log.info("No origDocId, calling createNewRecord")
albumItem = self.getAlbumItem()
return self.createNewRecord(cnt, item, albumItem) #objectId)
def defaultMimetypeId(self, maintype, level="document"):
if maintype == "audio": return "113"
if maintype == "video":
if level == "album":
return "122" # video/x-dvd
else:
return "112"
return "130" # application/x-iso-image
def createNewRecord(self, rn, dn, item, albumItem):
"""Create new DB entry for document (e.g. audio track) referenced by item (number: cnt),
and cp entry.
Called to execute action M: reMember Metadata
and from createNewCopy() if there is no entry yet.
Return document ID if successful else None.
"""
albumCnt = int((getText(albumItem, self.rnColumn) or '').strip() or 1)
docsToA = self.docsToA if self.dirToImport else None
title = getText(item, self.dbTitleColumn) or getText(item, self.extTitleColumn)
level = getText(item, self.levelColumn)
albumLevel = getText(albumItem, self.levelColumn)
#if level == "album":
## docDic = docsToA[0] if docsToA else None
#docDic = docsToA.getOrig() if docsToA else None
#else:
##docDic = docsToA[int(cnt)-1] if docsToA else None
#docDic = docsToA.getOrig(int(cnt)) if docsToA else None
docDic = docsToA.getOrig(int(rn) if level == "album" else 1) if docsToA else None
albumId = getText(albumItem, self.idColumn)
# print "docDic:", docDic
# print("albumLevel: %s" % albumLevel)
if albumLevel == "album":
objectId = albumId
elif albumLevel == "package":
objectId = dbq.getObjectOfSimpleProduct(albumId)
# print("objectId: %s" % objectId)
# Determine original's mimetype
type = self.defaultMimetypeId(self.maintype, level)
#if self.maintype == "audio":
#type = "113"
#elif self.maintype == "video":
#if level == "album": type = "122" # video/x-dvd
## elif level == "document": type = "112" # MP
#else: type = "112" # MP
#else: type = "130" # application/x-iso-image
# TODO: For imported files from LP or MC directory, put an 'A' or 'B' before name
# see takeTitlesAsFilenames
if level in ("album", "package"):
file = '/'
if self.maintype not in ("audio","video"): type = "111" # CS
elif self.maintype == "audio":
if self.discid:
file = "track%s.cdda" % rn
elif self.dirToImport:
file = docDic["file"] # TODO: split it up
#d = dbdocs[i] if len(dbdocs)>=i+1 else None
#e = docList[i] if len(docList)>=i+1 else None
#file = d["file"] if d and "file" in d else (e["file"] if e else '')
else: file = archive.createFilename(title) #:"part " + unicode(cnt)
if not objectId:
# TODO: Ensure there is an object
errmsg("objectId not given (1987)")
return None
ta = archive.FileDbTransaction("Archiving " + title)
try:
origdoc = document.Document()
origdoc.vd.update({"title":title, "mimetypeid":type, "licenseid":"4"})
ta.sqlList.append(origdoc.save())
docId = origdoc.getId()
ta.sqlList.extend(db.ensureRecordExists('cp',
{'objectid':objectId, 'documentid':docId, 'file':q(file)}))
except Exception as s: # rollback due to SqlError or file error
ta.exceptionHandling(s)
return None
ta.commit() # ta.confirmCommit()
id = objectId if file=='/' else docId
item.setText(self.idColumn, id)
item.setText(self.dbTitleColumn, title)
item.setText(self.fileColumn, file)
return docId
def registerExistingDoc(self): # TODO: reform and reactivate
"""Register existing documents as version of marked document.
"""
log = logging.getLogger("registerExistingDoc")
caption = "Register existing documents"
item = self.currentItem()
origDocId = self.ensureDocumentRecordExists()
# origDocId = h.getDocId(item)
# cnt = getText(item, h.rnColumn)
# if not origDocId:
# log.info("No origDocId, calling createNewRecord")
# self.createNewRecord(cnt, item)
# origDocId = h.getDocId(item)
log.info("origDocId: %s" % origDocId)
title = getText(item, h.dbTitleColumn) or getText(item, h.extTitleColumn)
crit = "title=%s AND id!=%s" % (q(title), origDocId)
f = finddialog.FindEntityAndTypeDialog('document')
#f.helpIntro = "<p>Please select or create a package</p>"
f.critEdit.setText(crit)
f.showCandidates()
if not f.exec_(): return
# sql = "INSERT INTO rel (ent1,id1,relid,ent2,id2) VALUES (5,%s,19,5,%s)" % (f.selectedId(), origDocId)
sql = "UPDATE document SET versionof=%s WHERE id=%s" % (origDocId, f.selectedId())
log.debug(sql)
db.execSql(sql)
def addMetadata(self):
"""Register metadata for product and document.
Currently handled are the following data:
- persons as actors, directors, composers, scriptwriters
- date (document.date and product.releasedate)
- rating in document.summary
- producer (label) as product.companyid
- product.ean
- document.review from metadata fields summary, summary2, summary3
- document.languageid
- document.length
"""
caption = "addMetadata"
log = logging.getLogger("addMetadata")
item = self.currentItem()
product = self.getProductWithEan(item)
if not product:
errmsg("No product with EAN", caption)
return
# Find the infosourcecontent entry of this product.
# TODO: prepend EAN with 0 if needed
# adapt to new FindDialog syntax
f = finddialog.FindEntityAndTypeDialog("infosourcecontent")
title = product["name"]
crit = "mainfield=%s" % q(title)
ean = product["ean"].strip()
if ean: crit += " OR publicid=%s" % q(ean)
f.critEdit.setText(crit)
f.showCandidates()
if not f.exec_(): return
id = f.selectedId()
mdr = metadatareader.MetadataReader(infosourcecontentid=id)
if not mdr.exec_(): return
origDocId = self.ensureDocumentRecordExists()
#log.info("origDocId: %s" % origDocId)
# Let user confirm the document.
f = finddialog.FindEntityAndTypeDialog("document",
msg = "Please select the document these metadata refer to.")
f.critEdit.setText("#%s" % origDocId)
f.showCandidates()
if not f.exec_(): return
docId = f.selectedId()
# Let user confirm the product.
f = finddialog.FindEntityAndTypeDialog("product",
msg = "Please select the product these metadata refer to.")
pid = product["id"]
if pid: f.critEdit.setText("#%s" % pid)
f.showCandidates()
if not f.exec_(): return
productId = f.selectedId()
log.info("docId: %s; productId: %s; registering roles (actors etc.)" % (docId, productId))
# TODO: Ask if data refer to this document or a prototype (original).
# TODO: If a person (actor etc.) is created, guess language from that of document.
getVal = metadatareader.getAttributeValue # Give function a handier name.
sep = getVal(mdr.attributes, "sep")
if not document.registerRoles(getVal(mdr.attributes, "actors"), sep, docId, 'actors'): return
if not document.registerRoles(getVal(mdr.attributes, "directors"), sep, docId, 'directors'): return
if not document.registerRoles(getVal(mdr.attributes, "composers"), sep, docId, 'composers'): return
if not document.registerRoles(getVal(mdr.attributes, "scriptwriters"), sep, docId, 'scriptwriters'): return
log.debug("registering date etc.")
rawdate = getVal(mdr.attributes, "date")
normdate = db.getDate(rawdate)
if normdate:
sql = "UPDATE document SET date='%s' WHERE id=%s" % (normdate, docId)
db.execSql(sql)
sql = "UPDATE product SET releasedate='%s' WHERE id=%s" % (normdate, productId)
db.execSql(sql)
rating = getVal(mdr.attributes, "rating")
sql = "UPDATE document SET summary=%s || COALESCE(summary, '') WHERE id=%s" % (q(rating), docId)
db.execSql(sql)
# getMainImage getVal(mdr.attributes, "imagepage")
producerText = getVal(mdr.attributes, "producer")
if producerText:
f = finddialog.FindEntityAndTypeDialog("company")
f.helpIntro = "Please select a company as producer."
f.critEdit.setText("name~*%s" % q(producerText))
f.newRecordValue = producerText
f.initDic = {"remark":"producer of %s" % title}
f.showCandidates()
if f.exec_() and f.selectedId():
sql = "UPDATE product SET companyid=%s WHERE id=%s" % (f.selectedId(), productId)
db.execSql(sql)
ean = getVal(mdr.attributes, "ean")
if ean:
log.debug("registering EAN: %s" % ean)
sql = "UPDATE product SET ean=%s WHERE id=%s" % (q(ean), productId)
db.execSql(sql)
genre = getVal(mdr.attributes, "genre")
if genre:
sql = "UPDATE document SET contype=%s WHERE id=%s" % (q(genre), docId)
db.execSql(sql)
text = getVal(mdr.attributes, "summary")
if text:
sql = "UPDATE document SET review=%s WHERE id=%s" % (q(text), docId)
db.execSql(sql)
text = getVal(mdr.attributes, "languages")
if text:
llist = db.parseLanguages(text)
if llist:
sql = "UPDATE document SET languageid=%s WHERE id=%s" % (llist[0], docId)
db.execSql(sql)
text = getVal(mdr.attributes, "length")
if text:
sql = "UPDATE document SET length='%s' WHERE id=%s" % (getLength(text), docId)
log.debug("registering length: %s" % sql)
db.execSql(sql)
url = getVal(mdr.attributes, "image")
if url:
# Download image, create document for it, create rel
log.debug("registering image from URL: %s" % url)
document.downloadImage(url, productId)
self.fillFields(productId=product["id"])
#self.productWidget.reread()
#self.docWidget.documentWidget.reread()
#log.debug("Done.")
def fetchDataByEan(self):
caption = "fetchDataByEan"
# log = logging.getLogger("fetchDataByEan")
item = self.currentItem()
product = self.getProductWithEan(item)
if not product:
errmsg("No product with EAN", caption)
return
ean = product["ean"]
if not ean: return
#msgbox("Fetching data for EAN: " + ean)
#print("i = infosource.Infosource(100)")
i = infosource.Infosource(100)
p = i.getPageByEan(ean)
idList = i.recordDetail(p,'product','ean')
msgbox("Ready fetching data for EAN: " + ean)
def fetchDataByTitle(self):
albumItem = self.getAlbumItem()
if albumItem:
title = getText(albumItem, self.dbTitleColumn)
else:
(text,ok) = QInputDialog.getText(self, "Enter title",
"Please enter title here.")
if not ok: return
title = unicode(text)
msgbox("Fetching data for title: " + title)
i = infosource.Infosource(128)
p = i.getPageByTitle(title,'any')
idList = i.recordDetail(p,'product','ean')
if not idList: msgbox("No data found"); return
crit = ' OR '.join(["id=%s" % id for id in idList])
f = finddialog.FindDialog("infosourcecontent")
f.critEdit.setText(crit)
if not f.exec_(): return
id = f.selectedId()
mdr = metadatareader.MetadataReader(infosourcecontentid=id)
if not mdr.exec_(): return
if not productId: return
p = product.Product()
p.readFromDb(productId)
p.getMetadataValues(mdr.attributes)
ta = archive.FileDbTransaction('') #caption)
try:
# DB and file operations
ta.sqlList = p.saveExtended() # calls doc.save() and gets id; returns SQL list
for relSql in p.relSqls:
ta.sqlList.append(db.execSql(relSql % p.vd["id"]))
except Exception as s: # rollback due to SqlError or file error
ta.exceptionHandling(s)
# else:
# if ta.confirmCommit():
# self.setId(product.vd["id"])
# Helper functions ==============================================
# def productDiscId(self):
# """Return discid from product DB record or None."""
# self.productWidget.getValue("discid")
def exportCopy(self, rn, dn, item, level, origDocId, title, filename,
destPath, destFormat, albumItem, destDir, multiProgress):
caption = "exportCopy"
log = logging.getLogger(caption)
log.debug("rn=%s, item=%s, destFormat=%s, albumItem=%s", rn, item, destFormat, albumItem)
docsToA = self.docsToA
if not self.setDevice(): return
src = None # path if source is file or directory
(documentId, path) = document.getDocumentVersion(origDocId)
log.debug("version documentId: %s; path: %s" % (documentId, path))
path = path or document.getAvailableFile(documentId)
if not path: return
(dirName, fileName) = os.path.split(path)
destPath = destDir + '/' + fileName
cmd = ["cp", path, destPath]
statinfo = os.stat(path)
endValue = statinfo.st_size
p = DestinationFileProcess(multiProgress, "Export " + fileName,
cmd, destPath, endValue)
p.startCmd()
print "after startCmd()"
#if e:
#errmsg(e) #, caption)
#return False
return True
# TODO:
# - How to calculate the uncompressed size? Is the compression factor known?
#def createNewCopy(self, rn, item, destFormat, albumItem, multiProgress):
def createNewCopy(self, rn, dn, item, level, origDocId, title, filename, destPath,
destFormat, albumItem, albumObjectId, albumTitle, albumDescr, albumCnt,
multiProgress):
"""Create new copy of document (e.g. audio track) referenced by item (number: cnt), in given format.
Not for registering existing documents.
albumItem is the item itself if it is an album, or
the parent (ancestor) album, if the item is a document.
Needed to find out which object a document belongs to,
even if the document is not yet in the DB.
Strictly speaking, this does several jobs:
- Ripping and converting is delegated to ripEncode()
- Copying/moving to destination
- Regrouping
One or more of these may be null.
class FileProcess
destPath
destSize estimate
progress
hash (copied from src or calculated)
size (of destPath)
origSize
compFactor
function can be:
dir -> isoFile
cp or mv file
rip file
dvdbackup to dir and then to isoFile
dd
mp3[un]wrap
tiffcp %s '%s'; tiff2pdf
convert
cdparanoia
genisoimage
parameters:
destPath, srcList, device
"""
#def ripEncode(self, rn, item, level, origDocId, title, filename, destPath,
#destFormat, albumItem, albumObjectId, albumTitle, albumDescr, albumCnt,
#multiProgress):
caption = "createNewCopy"
log = logging.getLogger(caption)
# docsToA = self.docsToA
# if not self.setDevice(): return
if not origDocId: # No original document record yet.
self.createNewRecord(rn, dn, item, albumItem)
origDocId = self.getDocId(item)
#origSize = None # size in bytes of original, e.g. audio track
#compFactor = 1 # typical compression factor of destFormat
#src = None # path if source is file or directory
log.debug(ds(locals(), "albumObjectId,albumCnt,albumTitle"))
(origDoc, copyDoc) = self.docsToA.getAlbumDocPair(albumCnt, level, dn)
log.debug(ds(locals(), "origDoc,copyDoc"))
# copyDoc is a single document
#if not copyDoc: # TODO
#errmsg("Nothing to copy")
#return
# if docDic and "copymtid" in docDic: srcFormat = mimetype.Mimetype(docDic["copymtid"])
hashSql = None
if self.discid:
success = self.ripEncode(rn, dn, item, level, origDocId, title, filename,
destPath, destFormat,
albumItem, albumObjectId, albumTitle, albumDescr, albumCnt,
multiProgress)
if not success: return False
self.scheduleSingleAction(item, destFormat.getId(), ' ') # remove action from schedule
# TODO: Show that new copy exists.
caption = "New Document: " + title
# Now file is in toarchive, so there is no precalc hash in rawfilelist!!!
doc = document.Document(); doc.src = destPath
if doc.checkDuplicate(): return
doc.vd.update({"title":title, "mimetypeid":destFormat.getId()}) #, "prototypic":"false"
doc.vd.update({"versionof":origDocId})
id = document.createDocumentRecord(doc, caption, confirm=False)
return True # success
if copyDoc and "mimetypeid" in copyDoc.vd:
srcFormat = mimetype.Mimetype(copyDoc.vd["mimetypeid"])
if self.dirToImport:
srcPath = copyDoc.src #docDic["albumDir"] + '/' + docDic["file"]
origSize = os.path.getsize(srcPath)
files = [d.src for d in docsToA.getCopy()]
exts = set([os.path.splitext(f)[1].lower() for f in files])
filetext = getText(item, self.fileColumn)
if self.dirToImport and (level == "document" or filetext == '/'):
if srcFormat.getId() == destFormat.getId():
cmd = "mv %s %s" % (qs(srcPath), qs(destPath)) # better use copyRename()? TODO
shell = True
# Copy precalc hash. TODO: Check if isodate is correct, see archive.copyAndRecord()
hashSql = textwrap.dedent("""
INSERT INTO rawfilelist (host, file, size, isodate, hash)
SELECT host, %s, size, isodate, hash
FROM rawfilelist
WHERE file=%s
""")[1:-1] % (q(os.path.realpath(destPath)), q(os.path.realpath(srcPath)))
print "hashSql", hashSql
elif self.dirToImport and level == "album" and exts == set([".mp3"]) and destFormat.getId() == '57':
srcList = ''; origSize = 0
for docDic in docList:
srcPath = docDic["albumDir"] + '/' + docDic["file"]
srcList += "'%s' " % srcPath
origSize += os.path.getsize(srcPath)
if len(docList) > 1:
cmd = "mp3wrap '%s' %s" % (destPath, srcList)
else:
cmd = "cp %s '%s'" % (srcList, destPath)
elif self.maintype == "text":
if not self.dirToImport or level != "album":
errmsg("Not yet supported")
return
if exts == set([".tif"]):
tmpfile = archive.getFreeToarchiveFilepath("tmpfile")
srcList = ''; origSize = 0
for docDic in docList:
srcPath = docDic["albumDir"] + '/' + docDic["file"]
srcList += "'%s' " % srcPath
origSize += os.path.getsize(srcPath)
if not srcList:
errmsg("No source file")
return
cmd1 = "tiffcp %s '%s'; tiff2pdf '%s'; rm '%s'" % (srcList, tmpfile, tmpfile, tmpfile)
cmd = "(%s) > '%s'" % (cmd1, destPath)
elif exts == set([".jpg"]):
tmpfile = archive.getFreeToarchiveFilepath("tmpfile")
srcList = ''; origSize = 0
for docDic in docList:
srcPath = docDic["albumDir"] + '/' + docDic["file"]
srcList += "'%s' " % srcPath
origSize += os.path.getsize(srcPath)
if not srcList:
errmsg("No source file")
return
cmd = "convert -monitor -verbose %s '%s'" % (srcList, destPath)
print("cmd:", cmd)
# return
elif self.maintype == "audio":
# if self.discid: archiving a CD/DVD
# if self.dirToImport: importing a directory
if self.discid:
if self.objectInDevice == albumObjectId:
# cmd1 is the command that reads (or rips) the file and writes it to stdout
# cmd2 is the command that reads the file from stdin and writes it in the destination format to dest
if level == "album":
cmd1 = "cdparanoia -Z -d %s 1- -" % (self.device)
origSize = getTrackSize(self.device, None)
print("origSize ", origSize)
elif level == "document":
# Without -Z, with bad CDs, it hangs forever.
cmd1 = "cdparanoia -Z -d %s %s -" % (self.device, rn)
origSize = getTrackSize(self.device, rn)
print("origSize ", origSize)
else:
msg = "Need %s in drive %s. Currently in drive is\n" % (albumDescr, self.device)
msg += db.descrWithId("object", self.objectInDevice) if self.objectInDevice else "no audio CD"
errmsg(msg)
return
elif self.dirToImport:
print("rn", rn)
if level == "album": # Convert to wav and concatenate; rn is album counter
origSize = 0 #getTrackSize(self.device, None)
#docList = self.albumList[int(rn)-1]
# This first converts to wav, then concatenates and finally
# in cmd2, converts to destination format.
# Other strategies are possible, e.g. if source is MP3:
# just use mp3wrap.
# Another strategy, for TIF->PDF:
# tiffcp, then tiff2pdf
# Need intermediate file for that, or named pipe; also for mplayer
# one command is a list of strings; for args: "%(srcPath)"
# there may be several commands
# there may be tmp.files, as many as src files
# is there an intermediate format (such as wav)?
# To summarize data per action:
# - list of commands, with placeholders for src and dest filenames
# - compression factor for progress calculation
# - how to detect errors (from exitcode or stderr)
# TODO: Murks hier, gründlich überarbeiten!
cmd1 = "("
for docDic in docList:
#docDic["albumDir"] = albumDir
srcPath = docDic["albumDir"] + '/' + docDic["file"]
print(srcPath)
origSize += os.path.getsize(srcPath)
cmd1 += "ffmpeg -i %s -f wav -; " % qs(srcPath) #(docDic["albumDir"] + '/' + docDic["file"])
cmd1 += ")"
elif level == "document": # Convert to wav; here, rn is file counter
docDic = self.albumList[albumCnt-1][int(rn)-1]
srcPath = docDic["albumDir"] + '/' + docDic["file"]
cmd1 = "ffmpeg -i '%s' -f wav -" % srcPath
origSize = os.path.getsize(srcPath)
origSize = origSize * 10 # compFactor of e.g. WMA TODO
print("origSize ", origSize)
print(cmd1)
else:
descs = getDescendantItems(h, item)
msg = ''
for i in descs:
msg += getItemDescr(h, i) + '\n'
msgbox(msg)
# msgbox("Creating new documents from archived ones is not yet supported", caption)
return
if destFormat.subtype() == "mpeg":
cmd2 = "lame - '%s'"
compFactor = 10
elif destFormat.subtype() == "x-ogg":
cmd2 = "oggenc - -o '%s'"
compFactor = 10
elif destFormat.subtype() == "x-aac":
cmd2 = "faac - -o '%s'"
compFactor = 10
elif destFormat.subtype() == "flac":
cmd2 = "flac - -o '%s'"
compFactor = 10
elif destFormat.subtype() == "x-ms-wma":
cmd2 = "ffmpeg -i - -acodec wmav2 -ab 128 '%s'"
compFactor = 10
pass # Conv. WMA to MP3: mplayer -vo null -vc dummy -af resample=44100 -ao pcm -waveheader $i && lame -m s audiodump.wav -o $i
# mplayer src.wma -ao pcm -ao pcm:file=file.wav
#cmd2 = "flac - -o '%s'"
#compFactor = 10
else:
errmsg("Audio format not yet supported: " + destFormat.subtype())
return
cmd2 = cmd2 % destPath
cmd = cmd1 + " | " + cmd2
shell = True
elif self.maintype == "video":
# If album is video DVD copy (i.e. a DVD playable in standalone player,
# but not original, so without CSS), then creating an ISO from the DVD,
# the same way as with (game etc.) CD-ROMs, is one method of creating a copy,
# maybe even the preferred one. To determine this, take into account:
# format.subtype()? origDoc etc.
print "self.discid=%s self.objectInDevice=%s self.objectId=%s" % (self.discid,self.objectInDevice,self.objectId)
if self.discid and self.objectInDevice == self.objectId:
if destFormat.subtype() == "mp4":
print "d = dvdinfo.HbDvd(self.device)"
d = dvdinfo.HbDvd(self.device)
print "discTitle = d.getDiscTitle()"
discTitle = d.getDiscTitle()
print "discTitle ", discTitle
ix = d.getLongestTitleIndex()
# print "getLongestTitleIndex", d.getLongestTitleIndex()
#ix = d.getTitleIndex(t)
print "getLongestTitleIndex ix", ix
#rawdest = "%s/%s.mp4" % (outputdir, discTitle)
dest = destPath #uniqueFileName(rawdest)
print "dest ", dest
# al = d.getAudio(title=t)
# cmd = "HandBrakeCLI -i /dev/sr0 -o /unibas_archive/raw/DVD/Ferkeltest.mp4 %s %s" % (opts_a, opts_s)
cmd = ["HandBrakeCLI", "-i", self.device]
cmd.extend(["-t", "%d" % ix])
cmd.extend(["-o", dest])
cmd.extend(d.getAudioOption())
cmd.extend(d.getSubtitleOption())
shell = False
outRegExpString = ".*Encoding: task \d+ of \d+, (\d+)\.\d+ %.*"
errRegExpString = "Encoding: task \d+ of \d+, (\d+)\.\d+ %"
print ' '.join(cmd)
# msgbox(' '.join(cmd))
# retcode = subprocess.call(cmd)
# return #(retcode, dest)
elif QMessageBox.question(None, caption, "Copy DVD dir structure with dvdbackup?",
QMessageBox.StandardButtons(QMessageBox.No | QMessageBox.Yes),
QMessageBox.No) == QMessageBox.Yes:
# Note: dvdbackup produces a directory; create iso file with:
# genisoimage -gui -dvd-video -o "%s" "%s"' % (dest, src)
cmd = "dvdbackup -M -i %s -o '%s'" % (self.device, destPath)
origSize = self.dvdSize if self.dvdSize else 8123456789
# not so simple; must be packed as ISO, errors must be detected
# even with -v option, there is no progress information
# but we can first list with -L all files and their sizes,
# then watch the copy files as they are created,
# calculate the total size of the copies and use this as progress indicator.
elif QMessageBox.question(None, caption, "Create ISO9660 image with dd?",
QMessageBox.StandardButtons(QMessageBox.No | QMessageBox.Yes),
QMessageBox.Yes) == QMessageBox.Yes:
cmd = "dd if='%s' of='%s'" % (self.device, destPath)
else:
msgbox("Currently no other format supported!")
# TODO dvdbackup
return
#elif self.filesToArchive:
elif self.dirToImport:
#msgbox("srcFormat.getId(): %s" % srcFormat.getId())
#if srcFormat.getId() == "125": # video DVD ISO
#cmd = "cp '%s' '%s'" % (srcPath, destPath)
if srcFormat.getId() == "122": # physical video DVD content
# eigentlich sehr unsauber; destFormat ist nur bekannt von sched. Action
# eigentlich könnte man aus 122 auch andere Formate produzieren
# ist also nur dann korrekt wenn ich hier 125 selektiere
#upcaseDvdDir(src)
#cmd = '/usr/bin/genisoimage -gui -dvd-video -o "%s" "%s"' % (dest, src)
#print "orig_src", origDoc.vd["src"]
#print "copy_src", copyDoc.vd["src"]
src = copyDoc.src #docDic["albumDir"]
#msgbox(src)
upcaseDvdDir(src)
# origSize = os.path.getsize(srcPath)
origSize = getDirSize(src); compFactor = 1
cmd = "/usr/bin/genisoimage -gui -dvd-video -o %s %s" \
% (qs(destPath), qs(src))
elif srcFormat.getId() in ("104","108","125","168","133"): # .avi or .ogv file or so
#cmd = "cp '%s' '%s'" % (srcPath, destPath)
cmd = "mv %s %s" % (qs(srcPath), qs(destPath)) # better use copyRename()?
# Copy precalc hash. TODO: Check if isodate is correct, see archive.copyAndRecord()
hashSql = textwrap.dedent("""
INSERT INTO rawfilelist (host, file, size, isodate, hash)
SELECT host, %s, size, isodate, hash
FROM rawfilelist
WHERE file=%s
""")[1:-1] % (q(os.path.realpath(destPath)), q(os.path.realpath(srcPath)))
print "hashSql", hashSql
#msgbox(hashSql)
else:
errmsg("Format not yet supported: %s" % db.descrWithId("mimetype",srcFormat.getId()))
return
else:
msg = "Wrong or no medium in device"
errmsg(msg, caption)
return
elif self.maintype == "application":
compFactor = 1
print("self.discid=%s self.objectInDevice=%s self.objectId=%s" % (self.discid,self.objectInDevice,self.objectId))
print "self.ot", self.ot
if self.ot == "CD":
origSize = 700000000
elif self.ot == "DVD":
origSize = 4700000000
if self.discid and self.objectInDevice == self.objectId:
# cmd = "dd if='%s' of='%s'" % (self.device, destPath)
cmd = ["dd", u"if={}".format(self.device), u"of={}".format(destPath)]
# shell = False # That's the default.
# Sometimes I get an error message here, don't know why.
# The output seems to be good, can be mounted, nothing missing.
#dd: reading `/dev/scd0': Input/output error
#1098976+0 records in
#1098976+0 records out
#562675712 bytes (563 MB ) copied, 140.734 s, 4.0 MB/s
#ProcessError: 5
else:
msg = "Need shown CD in drive %s. Currently in drive is\n" % self.device
msg += db.descrWithId("object", self.objectInDevice) if self.objectInDevice else "apparently no CD-ROM"
errmsg(msg)
return
else:
msgbox("Not yet implemented format copy: " + self.maintype)
return
print(cmd)
# TODO: progress feedback and interrupt possibility (use dd_rescue?)
# TODO: copy directly to destination? (not necessary if on same filesystem)
# TODO: create md5sum at the same time?
# TODO: Use startShellCmd
if hashSql:
#msgbox("now executing hashSql")
db.execSql(hashSql)
try:
# res = btoc(cmd)
# shellCmd = 'sh -c "%s"' % cmd
# print(shellCmd)
if origSize and compFactor:
endValue = origSize / compFactor
else:
endValue = 100 #5000000
# print "endValue ",endValue
#pp = ProcessProgress(shellCmd, windowTitle="Archiving file " + rn,
#labelText=destPath, endValue=endValue)
#pp.sizeFile = destPath
#pp.start()
#print "pp.success", pp.success
#if not pp.success:
#return False
# m.startShellCmd(shellCmd, destPath, endValue)
multiProgress.outRegExpString = outRegExpString
multiProgress.errRegExpString = errRegExpString
print "before startCmd"
multiProgress.startCmd(cmd, destPath, endValue, shell=shell)
print "after startCmd"
if not multiProgress.success:
return False
except Exception as s:
errmsg(unicode(s), caption)
return False
# Copy successful. TODO: Copy precalc hash from rawfilelist to filelist.
# Copy precalc hash is done in document.saveExtended()?
#If it was from file or dir, delete that. TODO: test and activate
#if self.filesToArchive: del self.filesToArchive[origDocId]
# Deletion must be done in a separate step.
# The user might want to take several different copies of the same source.
#if src:
#askAndRemove(src, "Delete source",
#"Copy succeeded, so source is no longer needed:\n%s\nDelete it?" % src)
self.scheduleSingleAction(item, destFormat.getId(), ' ') # remove action from schedule
# TODO: Show that new copy exists.
caption = "New Document: " + title
# Now file is in toarchive, so there is no precalc hash in rawfilelist!!!
doc = document.Document(); doc.src = destPath
if doc.checkDuplicate(): return
doc.vd.update({"title":title, "mimetypeid":destFormat.getId()}) #, "prototypic":"false"
doc.vd.update({"versionof":origDocId})
id = document.createDocumentRecord(doc, caption, confirm=False)
return True # success
# db.execSql("INSERT INTO rel (ent1,id1,relid,ent2,id2) VALUES (5,%s,19,5,%s)" % (id,origDocId))
#sh -c "(ffmpeg -i '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/01 Track 1.wma' -f wav -; ffmpeg -i '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/02 Track 2.wma' -f wav -; ffmpeg -i '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/03 Track 3.wma' -f wav -; ffmpeg -i '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/04 Track 4.wma' -f wav -; ffmpeg -i '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/05 Track 5.wma' -f wav -; ffmpeg -i '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/06 Track 6.wma' -f wav -; ffmpeg -i '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/07 Track 7.wma' -f wav -; ffmpeg -i '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/08 Track 8.wma' -f wav -; ) | lame - '/unibas_archive/raw/CD/ean9783866040649 Pu der Baer 3 CDs/CD 1/08 Track 8.wma'"
# TODO:
# - How to calculate the uncompressed size? Is the compression factor known?
def ripEncode(self, rn, dn, item, level, origDocId, title, filename, destPath,
destFormat, albumItem, albumObjectId, albumTitle, albumDescr, albumCnt,
multiProgress):
"""Create new copy of document in toarchive. Make no DB entries.
Maybe use for createNewCopy?
"""
# not needed: item, oridDocId, albumItem, albumObjectId, albumTitle, albumDescr, albumCnt
caption = "ripEncode"
log = logging.getLogger(caption)
if not self.setDevice(): return False
log.debug("self.maintype=%s", self.maintype)
if self.maintype == "audio":
cmd1data = {"album":('1-',''), "document":(dn,dn)}.get(level)
if not cmd1data:
errmsg("Ripping audio on level %s not yet supported." % level)
return False
cmd1 = "cdparanoia -Z -d %s %s -" % (self.device, cmd1data[0])
origSize = getTrackSize(self.device, cmd1data[1])
cmd2Dic = {"mpeg":("lame - '%s'",15), "x-ogg":("oggenc - -o '%s'",14),
"x-aac":("faac - -o '%s'",10), "flac":("flac - -o '%s'",2)}
cmd2data = cmd2Dic.get(destFormat.subtype())
if not cmd2data:
errmsg("Audio format not yet supported: " + destFormat.subtype())
return False
(cmd2, compFactor) = cmd2data
cmd = cmd1 + " | " + cmd2 % destPath
labelText = destPath.split('/')[-1]
p = DestinationFileProcess(multiProgress, labelText,
cmd, destPath, origSize/compFactor, shell=True)
elif self.maintype == "video":
# If album is video DVD copy (i.e. a DVD playable in standalone player,
# but not original, so without CSS), then creating an ISO from the DVD,
# the same way as with (game etc.) CD-ROMs, is one method of creating a copy,
# maybe even the preferred one. To determine this, take into account:
# format.subtype()? origDoc etc.
if self.discid and self.objectInDevice == self.objectId:
if destFormat.subtype() == "mp4":
d = dvdinfo.HbDvd(self.device)
discTitle = d.getDiscTitle()
ix = d.getLongestTitleIndex()
cmd = ["HandBrakeCLI", "-i", self.device]
cmd.extend(["-t", "%d" % ix, "-o", destPath])
cmd.extend(d.getAudioOption())
cmd.extend(d.getSubtitleOption())
outRegExpString = ".*Encoding: task \d+ of \d+, (\d+)\.\d+ %.*"
errRegExpString = "Encoding: task \d+ of \d+, (\d+)\.\d+ %"
print ' '.join(cmd)
p = PercentageProcess(multiProgress, "labelText", cmd,
outRegExpString=outRegExpString)
# msgbox(' '.join(cmd))
# retcode = subprocess.call(cmd)
# return #(retcode, dest)
elif self.maintype == "application":
origSize = {'CD':700000000, 'DVD':4700000000}.get(self.ot) or 4700000000
if self.discid and self.objectInDevice == self.objectId:
cmd = ["dd", u"if={}".format(self.device), u"of={}".format(destPath)]
p = DestinationFileProcess(multiProgress, "RipEncode " + destPath,
cmd, destPath, origSize) # shell=False by default
else:
msgbox("Not yet implemented format copy: " + self.maintype)
return False
res = p.startCmd()
if res:
errmsg(res, caption)
return False
return True
class AlbumAction():
"""Experimental.
3 cases:
1. From disc:
- CD: we know only titles
- DVD: propose dvdbackup, warn that size is big,
ask if full DVD or main feature (default full),
propose alternative DVDShrink of k9copy;
album with root doc only
2. From directory
We know things from dir and file names
3. From archive
We have data in DB already.
TODO: Test mp3wrap!
"""
def __init__(self, h, item, format, actionChar):
log = logging.getLogger("initAlbumAction")
#log.debug("rn=%s, format=%s, self.albumList=%s" % (rn, unicode(format), unicode(self.albumList)))
#caption = "createNewCopy"
# print "createNewCopy(rn, item, format):", rn, item, format
self.level = getText(item, h.levelColumn)
self.origDocId = h.getDocId(item)
title = getText(item, h.dbTitleColumn) or getText(item, h.extTitleColumn)
self.albumItem = h.getAncestorAlbumItem(item)
self.albumObjectId = getText(self.albumItem, h.idColumn) if self.albumItem else None
self.albumCnt = int(getText(self.albumItem, h.rnColumn)) if self.albumItem else None
docList = h.albumList[self.albumCnt-1] if h.albumList and self.albumCnt else None
#log.debug("albumCnt=%s; docList=%s; h.maintype=%s" % (self.albumCnt, unicode(docList), unicode(h.maintype)))
#if level=="album":
#self.docDic = docList[0] if docList else None
#else:
#self.docDic = docList[int(rn)-1] if docList else None
self.docDic = docList[0 if level=="album" else int(rn)-1] if docList else None
log.debug("docDic=%s" % docDic)
if self.docDic and "copymtid" in self.docDic:
self.srcFormat = mimetype.Mimetype(self.docDic["copymtid"])
if h.dirToImport:
self.srcPath = self.docDic["albumDir"] + '/' + self.docDic["file"]
origSize = os.path.getsize(self.srcPath)
exts = set([os.path.splitext(docDic["file"])[1].lower() for docDic in self.docList])
if h.dirToImport and self.level == "document":
if self.srcFormat.getId() == format.getId():
cmd = "cp '%s' '%s'" % (srcPath, destPath)
elif h.dirToImport and level == "album" and exts == set([".mp3"]) and format.getId() == '57':
srcList = ''; origSize = 0
for docDic in docList:
srcPath = docDic["albumDir"] + '/' + docDic["file"]
srcList += "'%s' " % srcPath
origSize += os.path.getsize(srcPath)
if len(docList) > 1:
cmd = "mp3wrap '%s' %s" % (destPath, srcList)
else:
cmd = "cp %s '%s'" % (srcList, destPath)
elif h.maintype == "text":
pass
class AlbumForm(QMainWindow):
def __init__(self, parent=None):
QMainWindow.__init__(self,parent)
self.setWindowTitle("Album Form")
self.setMinimumWidth(800)
self.topProductWidget = product.ProductWidget('0','0')
self.mainSplitter = QSplitter(Qt.Vertical, self)
self.mainSplitter.addWidget(self.topProductWidget )
self.tabWidget = QTabWidget(self) #, "tabWidget")
self.mainSplitter.addWidget(self.tabWidget)
self.setCentralWidget(self.mainSplitter)
self.contentsPage = QWidget() #self.tabWidget) #, "Main Page")
self.contentsLayout = QHBoxLayout()
h = self.hierarchyWidget = AlbumHierarchy(self.contentsPage)
h.form = self
h.objectId = None
h.copydir = None
h.maintype = None # "audio", "video" etc. for CD/DVD in drive
h.formats = []
h.formatColumn = {} # keys: mimetype IDs; values: column numbers
h.setSelectionMode(QAbstractItemView.ExtendedSelection)
h.defineBaseColumns()
h.setHeaderLabels(h.baseColumnNames) #["Level","Title"])
h.setColumnCount(h.columnCount)
self.contentsLayout.addWidget(h)
self.contentsPage.setLayout(self.contentsLayout)
self.tabWidget.addTab(self.contentsPage, "Contents")
self.productWidget = product.ProductWidget(None,'5') #None, "auto")
self.tabWidget.addTab(self.productWidget, "Product")
# self.objectWidget = object.ObjectWidget(None,'auto')
self.objectWidget = object.ObjectWithOwnerWidget()
self.tabWidget.addTab(self.objectWidget, "Object")
self.docWidget = document.DocumentWithRolesWidget()
self.tabWidget.addTab(self.docWidget, "Document")
#self.connect(self.objectWidget, SIGNAL("changedId"), self.setId)
self.connect(self.objectWidget.objectWidget, SIGNAL("changedProductId"), self.productWidget.setId)
self.connect(self.objectWidget.objectWidget, SIGNAL("changedProductId"), self.topProductWidget.setId)
self.connect(self.productWidget, SIGNAL("changedId"), self.topProductWidget.setId)
self.connect(self.hierarchyWidget, SIGNAL("changedDocId"), self.changedDocId)
self.connect(self.hierarchyWidget, SIGNAL("changedObjectId"), self.changedObjectId)
self.connect(self.hierarchyWidget, SIGNAL("changedProductId"), self.productWidget.setId)
self.setMenu()
#self.productWidget.setId(1)
self.docWidget.setId(None)
def changedDocId(self, id):
# if not id: return
self.docWidget.setId(id)
def changedObjectId(self, id):
self.objectWidget.setId(id)
def setMenu(self):
h = self.hierarchyWidget
fileMenu = self.menuBar().addMenu("&File")
fileMenu.addAction("Select medium", self.selectMedium)
fileMenu.addAction("Select medium via mediumid", self.selectMediumViaMediaid)
fileMenu.addAction("Select medium via product", self.selectProduct)
fileMenu.addAction("Select series", self.selectSeries)
fileMenu.addAction("Select medium via document", self.selectDocument)
#fileMenu.addAction("Import directory (&old)", self.importDirectory_old)
fileMenu.addAction("Import &directory", h.importDirectoryDialog)
fileMenu.addAction("Import &file album", h.importFileAlbum)
fileMenu.addAction("Reproduce album", h.reproduceAlbum)
#fileMenu.addAction("Commit import", self.commitImport)
fileMenu.addAction("E&xecute scheduled actions", self.execScheduledActions)
fileMenu.addAction("Check &barcode", h.checkBarcode)
fileMenu.addAction("&Check disc", h.checkDisc)
fileMenu.addAction("Check &audio CD", h.checkAudioCd)
fileMenu.addAction("Check &video DVD", h.checkVideoDvd)
fileMenu.addAction("addVideoDvd", h.addVideoDvd)
#fileMenu.addAction("Archive new video DVD directory", self.archiveDvdDir)
fileMenu.addAction("Prepare (tag) album", h.prepareAlbum)
fileMenu.addAction("createIsoFile", h.createIsoFile)
#fileMenu.addAction("Copy album to archive", self.copyAlbumToArchive)
#fileMenu.addAction("Create album copy on media", self.createAlbumCopy)
fileMenu.addAction("createMp3wrap", h.createMp3wrap)
fileMenu.addAction("Add &package", self.addPackage)
fileMenu.addAction("Open series", self.openSeries)
fileMenu.addAction("Close series", self.closeSeries)
fileMenu.addAction("&Quit", self, SLOT("close()")) #, CTRL+Key_Q )
fileMenu.addAction("&Save", self.save)
fileMenu.addAction("archivePartDoc", self.archivePartDoc)
editMenu = self.menuBar().addMenu("&Edit")
editMenu.addAction("Schedule action", self.scheduleAction)
editMenu.addAction("Take over external document titles", self.takeTitles)
editMenu.addAction("Take titles als file names", self.takeTitlesAsFilenames)
editMenu.addAction("Create &work", self.createWork)
editMenu.addAction("Adopt documents from product", self.adoptDocuments)
#editMenu.addAction("Register existing document as version", h.registerExistingDoc)
editMenu.addAction("Make document an episode of a series", self.makeEpisode)
editMenu.addAction("Remove episode from series", self.removeEpisode)
editMenu.addAction("Get external titles from FreeDB", self.getFreeDbTitles)
editMenu.addAction("Get external titles from file", self.getTitlesFromFile)
editMenu.addAction("Set owner", self.setOwner)
editMenu.addAction("makeRootDocs", self.makeRootDocs)
viewMenu = self.menuBar().addMenu("&View")
selectTabMenu = viewMenu.addMenu("Select &tab")
selectTabMenu.addAction("&Contents", self.focusContents)
selectTabMenu.addAction("&Product", self.focusProduct)
selectTabMenu.addAction("&Object", self.focusObject)
selectTabMenu.addAction("&Document", self.focusDocument)
viewMenu.addAction("&Refresh", self.refresh)
viewMenu.addAction("Add format column", self.selectAndAddFormatColumn)
viewMenu.addAction("Info", self.info)
# viewMenu.addAction("fillFields", self.fillFields)
viewMenu.addAction("showAlbumConnectionTree", self.showAlbumConnectionTree)
toolsMenu = self.menuBar().addMenu("&Tools")
toolsMenu.addAction("&proposeNextAction", self.proposedActionLoop)
toolsMenu.addAction("&Test", self.test)
toolsMenu.addAction("E&ject and Clear", self.ejectAndClear)
toolsMenu.addAction("&Clear", self.clearContents)
toolsMenu.addAction("Add downloaded metadata", h.addMetadata)
toolsMenu.addAction("Fetch data from infosource by EAN", h.fetchDataByEan)
toolsMenu.addAction("Fetch data from infosource by title", h.fetchDataByTitle)
toolsMenu.addAction("readDirInfosources", readDirInfosources)
# File menu ============================================================
def selectMedium(self):
"""Select medium (album object) and display its data."""
f = finddialog.FindDialog("object")
if not f.exec_(): return
objectId = f.selectedId()
if objectId: self.hierarchyWidget.fillFields(objectId=objectId)
def selectMediumViaMediaid(self):
caption = "Find medium via owner's mediaid"
(mediaid, ok) = QInputDialog.getText(self, caption, "Please enter the mediaid",
QLineEdit.Normal, '')
if not ok: return
sql = "SELECT objectid FROM owner WHERE mediaid=%s" % q(mediaid)
objectidCSV = ','.join(db.valueList(sql))
if not objectidCSV:
msgbox("No media with this mediaid found: %s" % mediaid)
return
f = finddialog.FindEntityAndTypeDialog("object")
f.critEdit.setText("!id IN (%s)" % objectidCSV)
f.showCandidates()
if not f.exec_(): return
objectId = f.selectedId()
if objectId: self.hierarchyWidget.fillFields(objectId=objectId)
def selectProduct(self, productId=None):
"""Select product which may be package, and display its data."""
fp = finddialog.FindDialog("product")
if not fp.exec_(): return
productId = fp.selectedId()
if productId: self.hierarchyWidget.fillFields(productId=productId)
def selectSeries(self):
"""Select product, then corresponding medium (object) and display its data."""
seriesId = document.findSeries()
if seriesId: self.hierarchyWidget.fillFields(seriesId=seriesId)
def selectDocument(self):
"""Select document, then corresponding medium (object) and display its data."""
fp = finddialog.FindDialog("document")
if not fp.exec_(): return
documentId = fp.selectedId()
objectId = dbq.getAlbum(documentId)[0]
if objectId:
self.hierarchyWidget.fillFields(objectId=objectId)
return
documentId = db.readSql("SELECT versionof FROM document WHERE id=%s" % documentId, 0, 1)
if not documentId:
msgbox("No album for this document")
return
objectId = dbq.getAlbum(documentId)[0]
if objectId:
self.hierarchyWidget.fillFields(objectId=objectId)
return
msgbox("No album for this document")
#def commitImport(self):
#""" !!! TODO: Also remove album and package.
#Execute scheduled operations, prepared by importDirectory().
#NOT for actions scheduled with Edit - Schedule action. See File -Execute scheduled actions.
#TODO: Take care of *_MP3WRAP.mp3
#id3 tags
#"""
#log = logging.getLogger("commitImport")
#h = self.hierarchyWidget
#dir = h.dirToImport
#caption = "Commit import of directory: %s" % dir
#if h.filesToArchive is None:
#errmsg("No files defined to archive. Call File - Import Directory first.", "Album Form")
#return
#ta = archive.FileDbTransaction("Commit import")
#log.debug("h.filesToArchive: %s" % unicode(h.filesToArchive))
#for origDocId,v in h.filesToArchive.items():
#print(origDocId, v)
#(path, hash, mimetypeId) = v
#log.info("copy data: path: %s, mimetypeId: %s" % (path, mimetypeId))
#dl = db.dicList("SELECT * FROM document WHERE id="+origDocId)
#if not dl:
#errmsg("No entry for doc " + origDocId)
#return
#title = dl[0]["title"]
#try:
#origdoc = document.Document()
#origdoc.readFromDb(origDocId)
#origTypeId = origdoc.vd["mimetypeid"]
#log.info("orig mimetypeId: %s" % origTypeId)
#copydoc = copy.deepcopy(origdoc)
#copydocid = copydoc.getCloneId()
#copydoc.src = path
#copydoc.vd["hash"] = hash
#copydoc.vd["mimetypeid"] = mimetypeId
## m = mimetype.mimetypeFromFile(path) if path else None
## if m: copydoc.vd["mimetypeid"] = m.getId()
## TODO: Create filelist entry at the same time.
#if origTypeId == "122":
#log.info("Creating ISO file...")
## TODO: More general solution to create a file from a directory with several files,
## e.g.: genisoimage, mp3wrap, convert (JPG files to PDF), ...; see askAndWrapAlbum
#ta.crList.append(copydoc.createIso())
#log.info("Created ISO file from %s in %s" % (copydoc.src, copydoc.dest))
#else:
#log.info("copyRename")
#ta.crList.append(copydoc.copyRename())
#ta.sqlList.extend(copydoc.saveExtended()) # also creates cp entry and version rel entry
#except Exception as s: # rollback due to SqlError or file error
#log.error("Exception: %s" % s)
#ta.exceptionHandling(s)
#return
#ta.confirmCommit()
#db.execSql("select nextval('document_id_seq'::regclass);")
## If h.dirToImport is empty, remove it, else warn.
#for root, dirs, files in os.walk(dir, topdown=False):
#if files:
#msgbox("Directory %s not empty, can't remove it." % root, caption)
#return
#for subdirname in dirs:
#os.rmdir(os.path.join(root, subdirname))
#os.rmdir(dir)
def addPackage(self):
"""Create package from user's inputs. Parts will be added later."""
h = self.hierarchyWidget
caption = "Create new package"
msg = textwrap.dedent("""
<html><b>Creating a package of media of the same type</b><br><br>
Here you can enter a new package (e.g. of 2 CDs).<br>
First you enter the title of the package, <br>
then the medium type (e.g. 'CD'),<br>
then the number of media (e.g. 2).<br><br>
The program will then create a product entry from the title<br>
with the appropriate object type (e.g. ot='2 CDs').<br><br>
Subsequently you can enter the parts of the package.<br><br>
Please enter the package title now.</html>
""")[1:-1]
ds = h.lastPackageDataset
(name, ok) = QInputDialog.getText(self, caption, msg, QLineEdit.Normal, ds[0])
if not ok: return
(ean, ok) = QInputDialog.getText(self, caption, "Please enter EAN (or leave blank)", QLineEdit.Normal, ds[1])
if not ok: return
(part_ot, ok) = QInputDialog.getText(self, caption, "Please enter medium type. e.g. CD or DVD",
QLineEdit.Normal, ds[2])
if not ok: return
(num, ok) = QInputDialog.getInteger(self, caption, "Please enter number of %ss in this package" % part_ot,
ds[3], 1, 99, 1)
if not ok: return
msg = textwrap.dedent("""
<p>Please select recording type if appropriate.</p>
<p><dl>
""")[1:-1]
msgBox = QMessageBox()
msgBox.setWindowTitle(caption)
appButton = msgBox.addButton("Application", QMessageBox.ActionRole)
audioButton = msgBox.addButton("Audio", QMessageBox.ActionRole)
videoButton = msgBox.addButton("Video", QMessageBox.ActionRole)
noneButton = msgBox.addButton("None/Other", QMessageBox.ActionRole)
if part_ot == 'CD':
msgBox.setDefaultButton(audioButton)
elif part_ot == 'DVD':
msgBox.setDefaultButton(videoButton)
else:
msgBox.setDefaultButton(noneButton)
msgBox.exec_()
if msgBox.clickedButton() == appButton:
rt = 'Application'
elif msgBox.clickedButton() == audioButton:
rt = 'Audio'
elif msgBox.clickedButton() == videoButton:
rt = 'Video'
else:
rt = ''
h.lastPackageDataset = (name, ean, part_ot, num)
dic = {"name":name, "ot":"%d %ss" % (num, part_ot), "ean":ean, "rt":rt}
id = db.insertReturningId(db.createInsertReturningIdSql("product", dic))
#print "id:", id
# Show package.
instancewidget.showEntityForm(self, "product", id)
def save(self):
"""Save dbTitleColumn content of documents and works to document table.
"""
h = self.hierarchyWidget
for item in traverseRecursive(h.invisibleRootItem()):
level = unicode(item.text(h.levelColumn))
id = getText(item, h.idColumn)
if not id or level not in ("document","work"): continue
sql = db.createUpdateSql("document", {"id":id, "title":getText(item, h.dbTitleColumn)})
db.execSql(sql)
# Save column widths
for i in range(len(h.baseColumnWidths)):
h.baseColumnWidths[i] = h.columnWidth(i)
def makeRootDocs(self):
"""Turn selected docs into root docs.
"""
h = self.hierarchyWidget
for item in traverseRecursive(h.invisibleRootItem()):
if item.isSelected():
level = unicode(item.text(h.levelColumn))
if level not in ("document","work"): continue
id = getText(item, h.idColumn)
print "id:", id
docId = h.getDocId(item)
print "docId:", docId
if not docId:
errmsg("No document ID!")
return
cpids = db.valueList("SELECT id FROM cp WHERE documentid=%s" % docId)
if len(cpids) > 1:
errmsg("There are several copies!")
return
if len(cpids) != 1:
errmsg("Strange, no copy!")
return
sql = "UPDATE cp SET file='/' WHERE id=%s" % cpids[0]
print sql
db.execSql(sql)
self.refresh()
def archivePartDoc(self):
"""
- let user select document (or new file to archive it as in newDocumentFromFile)
- add origdoc and origdoc-cp entries
"""
h = self.hierarchyWidget
objectId = h.objectId # maybe remember others as well?
if not objectId:
msgbox("No object ID")
return
#documentId = document.newDocumentFromFile()
f = finddialog.FindEntityAndTypeDialog("document")
#f.showCandidates()
if not f.exec_(): return
documentId = f.selectedId()
if not documentId: return
copyDoc = db.idDic("document", documentId)
# TODO: function that creates appropriate origDoc from copyDoc
# also usable in importDir
origDoc = document.Document(initDic=copyDoc)
del origDoc.vd["hash"]
del origDoc.vd["size"]
mimetypeId = origDoc.vd["mimetypeid"]
if mimetypeId:
mt = mimetype.Mimetype(mimetypeId)
#if mt.maintype() == "audio": origDoc.vd["mimetypeid"] = '113'
#elif mt.maintype() == "video": origDoc.vd["mimetypeid"] = '112'
origDoc.vd["mimetypeid"] = self.defaultMimetypeId(mt.maintype())
#print origDoc
origDoc.save()
print "new origdoc id:", origDoc.vd["id"]
sql = "UPDATE document SET versionof=%s WHERE id=%s" % (origDoc.vd["id"], documentId)
print sql
db.execSql(sql)
sql = "INSERT INTO cp (objectid, documentid) VALUES (%s, %s)" % (objectId, origDoc.vd["id"])
print sql
db.execSql(sql)
# Edit menu ============================================================
def scheduleAction(self):
h = self.hierarchyWidget
if len(h.formats) == 0:
msgbox("There is no format yet. Please add a format column first.")
return
itemList = []
for item in traverseRecursive(h.invisibleRootItem()):
if item.isSelected():
itemList.append(item)
if not itemList:
msgbox("To schedule an action for an item, please select the item first!")
return
# defines actionChars
#X: export a copy
#V: register existing document as Version
#d: dummy action
listText = textwrap.dedent("""
R: Rip and encode only
N: create New copy: rip, encode and copy to archive
M: create DB record (reMember Metadata)
E: Export document
-: no action, or remove scheduled action
""")[1:-1]
actionList = listText.split('\n')
s = StringSelector(actionList)
if not s.exec_(): return
actionChar = s.selection()[0][0]
if actionChar == '-': actionChar = ' '
# Need formatId even in case of no action, to un-schedule
# a previously scheduled action.
if len(h.formats) == 1:
formatId = h.formats[0].getId()
else:
formatList = []
# Create format string list and try to preselect correct format.
# TODO: Thoroughly test this.
try:
d = h.albumList[0][0][0]
mt = d.vd.get("mimetypeid")
except:
mt = None
if mt == "122": mt = "125" # x-dvd -> x-video-dvd-image
for f in h.formats:
e = "%s: %s" % (f.getId(), f.vd["shortname"])
if f.getId() == mt:
formatList.insert(0, e)
else:
formatList.append(e)
# formatList = ["%s: %s" % (f.getId(), f.vd["shortname"]) for f in h.formats]
s = StringSelector(formatList)
if not s.exec_(): return
res = s.selection()[0]
formatId = getId(res)
#col = h.formatColumn[formatId]
for item in itemList:
h.scheduleSingleAction(item, formatId, actionChar)
def takeTitles(self):
"""Overwrite current document titles with those from external source (in extTitleColumn).
"""
h = self.hierarchyWidget
for item in traverseRecursive(h.invisibleRootItem()):
level = getText(item, h.levelColumn)
if level != "document": continue
extTitle = getText(item, h.extTitleColumn)
item.setText(h.dbTitleColumn, extTitle)
id = getText(item, h.idColumn)
sql = "UPDATE document SET title=%s WHERE id=%s" % (q(extTitle), id)
db.execSql(sql)
def takeTitlesAsFilenames(self):
#For imported files from LP or MC directory, put an 'A' or 'B' before name
h = self.hierarchyWidget
for item in traverseRecursive(h.invisibleRootItem()):
level = getText(item, h.levelColumn)
if level != "document": continue
title = getText(item, h.dbTitleColumn) or getText(item, h.extTitleColumn)
item.setText(h.fileColumn, title)
id = getText(item, h.idColumn)
if not id: continue
sql = "UPDATE cp SET file=%s WHERE documentid=%s" % (q(title), id)
print(sql)
db.execSql(sql)
# self.fillFields()
def createWork(self):
"""For selected documents, create container work and part rels.
Container work is prototypic, an audio work is sound only (SO).
MP3 (MP3WRAP) can additionally exist as its version.
This way the container work, as it is prototypic, it can be official, public,
and has no bias toward a specific format (others may prefer OGG or whatever).
In a proper data structure, the container work is needed to refer to by role records.
Playback of a container work can be done as playback of its archivable version,
or as playback of archivable versions of its parts.
TODO: Calc work title as longest common prefix, part titles as rest
"""
self.save()
h = self.hierarchyWidget
itemList = []; titleList = []
for item in traverseRecursive(h.invisibleRootItem()):
if item.isSelected():
title = getText(item, h.extTitleColumn) or getText(item, h.dbTitleColumn)
if not title or not getText(item, h.idColumn): #List[0]
errmsg("To create a work, every part must have a title and a document record (ID)")
return
itemList.append(item)
titleList.append(title)
if not itemList:
msgbox("To create a container work, please select parts first!")
return
title = commonPrefix(titleList)
# Prevent the 'A' in 'Adagio', 'Allegro', 'Andante' from being misunderstood as part of the title,
# if a work has only parts that begin with 'A'.
if len(title)>2 and title[-1] in 'Aa' and title[-2] in ' /:-,.':
title = title[:-2]
prefixLen = len(title)
if not title:
title = getText(itemList[0], h.extTitleColumn) or getText(itemList[0], h.dbTitleColumn)
firstPartId = getText(itemList[0], h.idColumn)
dic = db.dicList("SELECT summary, contype, date, mimetypeid, licenseid, true as prototypic, languageid "
+ "FROM document WHERE id=" + firstPartId)[0]
while title[-1] in ' /:-,': # Remove trailing punctuation that was between work and part
title = title[:-1]
dic.update({"title":title})
#print dic
sql = db.createInsertReturningIdSql('document', dic)
# "INSERT INTO document " \
# + "SELECT title, summary, contype, date, mimetypeid, licenseid, true as prototypic, languageid " \
# + "FROM document WHERE id=%s RETURNING id" % firstPartId
#print sql
workId = db.insertReturningId(sql)
#print workId
for (i0, item) in enumerate(itemList):
docId = getText(item, h.idColumn)
sql = "INSERT INTO rel (ent1, id1, ord, relid, ent2, id2) VALUES (%d, %s, %d, %d, %d, %s)" \
% (5, docId, i0+1, 44153, 5, workId)
db.execSql(sql)
sql = "UPDATE document SET title=%s WHERE id=%s" % (q(titleList[i0][prefixLen:].strip()), docId)
db.execSql(sql)
self.refresh()
def refresh(self):
h = self.hierarchyWidget
objectId = h.objectId # maybe remember others as well?
self.clearContents()
h.fillFields(objectId=objectId)
def adoptDocuments(self):
"""If there are other objects of the same product,
and those objects have cp entries,
adopt these entries."""
msg = ""
productId = self.productWidget.getId()
if not productId: msg = "No product."
#errmsg("No product")
#return
objectId = self.objectWidget.getId()
#if not objectId:
#errmsg("No object")
#return
if productId and objectId:
sql = "SELECT * FROM object WHERE productid=%s AND id!=%s" % (productId, objectId)
dl = db.dicList(sql)
if len(dl) == 0: msg = "No (other) object of this product"
#errmsg("No other object of this product")
#return
# There are products with the same contents (=same discid) but different EANs.
if msg: # Ask user to choose a product; default criterium: same discid.
if productId:
discid = self.productWidget.getValue("discid")
crit = "discid=%s" % q(discid) if discid else ""
f = finddialog.FindEntityAndTypeDialog("product")
f.helpIntro = "<p>%s Please select a (different) product</p>" % msg
f.critEdit.setText(crit)
f.showCandidates()
if not f.exec_(): return
productId = f.selectedId()
if not productId: return
sql = "SELECT * FROM object WHERE productid=%s" % (productId)
dl = db.dicList(sql)
if len(dl) == 0:
errmsg("No object of this product")
return
descrList = []; detailList = []; sourceId = None
for dic in dl:
sql = "SELECT SUBSTR(file,6,2) || ': ' || title as c FROM v_doc WHERE objectid=%s ORDER BY file;" % dic["id"]
print(sql)
content = ', '.join(db.valueList(sql))
print(objectId, content)
if content:
sourceId = dic["id"]
descrList.append("%s: %s" % (sourceId, dic["descr"]))
detailList.append(content)
if not sourceId:
errmsg("Sorry, no object has cp entries")
return
if len(descrList) > 1:
s = StringDetailSelector(descrList, detailList)
if not s.exec_(): return
sourceId = getId(s.selection()[0])
sql = "INSERT INTO cp (documentid, objectid, file) SELECT documentid, %s, file FROM cp WHERE objectid=%s ORDER BY file;" % (objectId, sourceId)
# print sql
db.execSql(sql)
self.refresh()
def openSeries(self):
"""Select or create a series document.
The following albums will be expected to be episodes of this series.
No more than one series can be "open" at a time.
When AlbumForm is closed, the series is automatically closed (forgotten).
To close a series explicitly in an AlbumForm session, closeSeries must be called.
TODO:
If a new album is to be registered and a series is open,
ask the user if this is the next episode of the open series.
Find the number n of the last registered episode (from rel.ord?)
and propose n+1 as the default episode number.
If the user agrees that this is an episode, remember it.
In scheduleDefaultActions, if this is an episode, propose to
register the whole album as one document.
Create a rel between album origdoc and series doc.
See node 3947: Document series
"""
caption = "Open series"
h = self.hierarchyWidget
#item = self.getAlbumItem()
#objectId = getText(item, h.idColumn)
#rootSql = "SELECT d.id FROM document d, cp WHERE d.id=cp.documentid AND objectid=" + objectId
#rootDocId = db.readSql(rootSql) # Root document ID is read from DB.
#seriesSql = "SELECT id2 FROM rel WHERE relid=63832 AND ent1=5 AND ent2=5 GROUP BY id2 ORDER BY id2 DESC LIMIT 20;"
#crit = "!id IN (%s)" % ','.join(db.valueList(seriesSql))
f = finddialog.FindEntityAndTypeDialog('document')
# f.helpIntro = "<p>Please select or create a package</p>"
# f.critEdit.setText(crit)
# f.order = "id DESC"
# f.newRecordValue = packageTitle
# f.initDic = package
# f.showCandidates()
if not f.exec_(): return
h.series = f.selectedId()
def closeSeries(self):
h = self.hierarchyWidget
h.series = None
def makeEpisode(self):
"""Make document (root doc of album or package) episode of a series.
See node 3947: Document series for a definition of series.
"""
caption = "Turn document into episode of a series"
h = self.hierarchyWidget
#item = self.getAlbumItem()
item = h.currentItem()
level = getText(item, h.levelColumn)
if level not in ("package", "album"):
msg = textwrap.dedent("""
This is only possible for album or package root documents.
In the rare case of a document from a package is an episode,
usually the package as a whole is the episode,
not a part album of it.
So in case of a single album, select that,
and if there is a package, usually select the package,
and try again.""")
msgbox(msg, caption)
return
rootDocId = h.getRootDocId(item)
# rootSql = "SELECT d.id FROM document d, cp WHERE d.id=cp.documentid AND objectid=" + objectId
# rootDocId = db.readSql(rootSql) # Root document ID is read from DB.
if not rootDocId:
# TODO: idea is good, implementation BROKEN!
# cp gets productid instead of objectid
if level == "package":
packageId = getText(item, h.idColumn)
descr = getText(item, h.dbTitleColumn)
dic = {"descr":descr, "productid":packageId}
objectId = db.createOrFindRecord("object", dic, "productid=%s" % packageId)
if not objectId: return
else:
objectId = getText(item, h.idColumn)
msg = "There is no root document yet. Create?"
if QMessageBox.question(None, caption, msg,
QMessageBox.StandardButtons(QMessageBox.No | QMessageBox.Yes),
QMessageBox.Yes) == QMessageBox.No: return
rootDocId = h.createNewRecord("01", "01", item, item)
if not rootDocId: return
seriesSql = "SELECT id2 FROM rel WHERE relid=63832 AND ent1=5 AND ent2=5 GROUP BY id2 ORDER BY id2 DESC LIMIT 20;"
crit = "!id IN (%s)" % ','.join(db.valueList(seriesSql))
f = finddialog.FindEntityAndTypeDialog('document', msg="Please select series")
# f.helpIntro = "<p>Please select or create a package</p>"
f.critEdit.setText(crit)
f.order = "id DESC"
# f.newRecordValue = packageTitle
# f.initDic = package
f.showCandidates()
if not f.exec_(): return
seriesId = f.selectedId()
maxOrdSql = "SELECT max(ord) FROM rel WHERE relid=63832 AND ent1=5 AND ent2=5 AND id2=" + seriesId
maxOrd = db.readSql(maxOrdSql)
default = int(maxOrd)+1 if maxOrd else 0
(ord, ok) = QInputDialog.getInteger(None, caption, "Please enter order in series", default, 0, 9999, 1)
if not ok: return
relDic = {"ent1":"5", "id1":rootDocId, "ord":ord, "relid":"63832", "ent2":"5", "id2":seriesId}
db.insertDic("rel", relDic)
def removeEpisode(self):
"""Remove episode document from series.
"""
caption = "Remove document from series"
h = self.hierarchyWidget
item = h.currentItem()
level = getText(item, h.levelColumn)
if level not in ("package", "album"):
msg = textwrap.dedent("""
This is only possible for album or package root documents.
In the rare case of a document from a package is an episode,
usually the package as a whole is the episode,
not a part album of it.
So in case of a single album, select that,
and if there is a package, usually select the package,
and try again.""")
msgbox(msg, caption)
return
rootDocId = h.getRootDocId(item)
if not rootDocId:
msgbox("No root doc")
return
removeSql = "DELETE FROM rel WHERE relid=63832 AND ent1=5 AND ent2=5 AND id1=%s" % rootDocId
db.execSql(removeSql)
id = getText(item, h.idColumn)
if level == "album":
h.fillFields(objectId=id)
else:
h.fillFields(productId=id)
def getFreeDbTitles(self):
"""Get album and track titles from FreeDB.
User needs to manually provide discid.
TODO: generalize
"""
caption = "getFreeDbTitles"
h = self.hierarchyWidget
discid = h.discid
if not discid: discid = self.productWidget.getValue("discid")
msg = "Please enter or confirm the discid:"
(discid, ok) = QInputDialog.getText(None, caption, msg, QLineEdit.Normal, discid)
if not ok: return
(genre,ok) = QInputDialog.getText(self, caption, "Enter Genre (misc, rock, data etc.)",
QLineEdit.Normal, "misc")
if not ok: return
#discid = unicode(qdiscid)
fbase = "http://www.freedb.org/freedb/%s/%s" % (unicode(genre), unicode(discid))
opener1 = urllib2.build_opener()
opener1.addheaders = [('User-agent', 'Mozilla/5.0')]
try:
f = opener1.open(fbase)
except urllib.error.HTTPError:
errmsg("HTTPError")
return
s = unicode(f.read(),"latin1") #"utf-8")
f.close()
ttitle = {}
for line in s.splitlines():
if line.startswith("DTITLE="):
dtitle = line[7:]
elif line.startswith("TTITLE"):
parts = line.split('=')
i = int(parts[0][6:])
# print i
ttitle[i] = parts[1]
#print dtitle
#print ttitle
for item in traverseRecursive(h.invisibleRootItem()):
level = getText(item, h.levelColumn)
rn = getText(item, h.rnColumn)
if level == "album": # and getText(item, h.fileColumn) == '/':
item.setText(h.extTitleColumn, dtitle)
elif level == "document":
item.setText(h.extTitleColumn, ttitle[int(rn)-1])
def getTitlesFromFile(self):
"""Get track titles from file.
"""
caption = "getTitlesFromFile"
h = self.hierarchyWidget
path = archive.selectToarchiveFile()
#path = "/tmp/tracks"
f = open(path, 'r')
s = unicode(f.read(),"utf-8")
f.close()
lines = []
#i = 1
for line in s.splitlines():
lines.append(line)
#print i, line
#i += 1
for item in traverseRecursive(h.invisibleRootItem()):
level = getText(item, h.levelColumn)
rn = getText(item, h.rnColumn)
try:
c = int(rn)
except:
continue
if level == "document":
if c > len(lines):
print "No input line for track %d" % c
else:
item.setText(h.extTitleColumn, lines[c - 1])
def setObjectOwner(self, objectId):
# log.info("Finding or creating owner of object %s" % objectId)
h = self.hierarchyWidget
ownerEntries = db.dicList("SELECT * FROM owner WHERE objectid=%s" % objectId)
if ownerEntries:
msgbox("This object already has one or more owner entries")
return
sql = "SELECT * FROM owner WHERE posdate IS NOT NULL ORDER BY posdate DESC, id DESC LIMIT 1;"
dl = db.dicList(sql)
if not dl:
msgbox("No default")
return
d = dl[0]
dic = {"posdate":isoNow()[:10], "objectid":objectId}
if d["personid"]: dic.update({"personid":d["personid"]})
if d["companyid"]: dic.update({"companyid":d["companyid"]})
if h.mediaid: dic.update({"mediaid":str(h.mediaid)})
print(dic)
db.insertDic("owner", dic)
# TODO: Refresh and focus
def setPackageOwner(self):
caption = "Set owner of package"
log = logging.getLogger("setPackageOwner")
h = self.hierarchyWidget
item = h.currentItem()
# sql = "SELECT packageid FROM product p, object o WHERE p.id=o.productid AND o.id=%s"
packageId = getText(item, h.idColumn)
log.info("Creating or finding object for package %s" % packageId)
descr = getText(item, h.dbTitleColumn)
dic = {"descr":descr, "productid":packageId, "orig":"true"}
objectId = db.createOrFindRecord("object", dic, "productid=%s" % packageId)
if not objectId: return None
self.setObjectOwner(objectId)
return objectId
def setOwner(self):
"""Set owner of current album. Take owner of last album as default
and switch to object widget."""
caption = "Set owner of album"
log = logging.getLogger("setOwner")
h = self.hierarchyWidget
#item = self.getAlbumItem()
item = h.currentItem()
if getText(item, h.levelColumn) not in ("album", "package"):
msgbox("Please select an album or package for this.")
return
if getText(item, h.levelColumn) == "package":
objectId = self.setPackageOwner()
else:
objectId = getText(item, h.idColumn)
self.setObjectOwner(objectId)
if objectId:
h.fillFields(objectId=objectId)
self.focusObject()
def getRootDocItemList(self):
"""Return list of items of album level where file='/'."""
h = self.hierarchyWidget; res = []
for item in traverseRecursive(h.invisibleRootItem()):
level = getText(item, h.levelColumn)
if level == "album" and getText(item, h.fileColumn) == '/':
res.append(item)
return res
def getToplevelItemList(self):
h = self.hierarchyWidget; res = []
for i in range (h.topLevelItemCount()):
res.append(h.topLevelItem(i))
return res
def getCurrentItemList(self):
h = self.hierarchyWidget; res = []
for item in traverseRecursive(h.invisibleRootItem()):
cur = getText(item, h.curColumn)
if cur == ' ': continue
res.append(item)
return res
def getScheduledActionList(self):
"""Return list of tuples (item, format, actionChar).
Used to find out if and which actions are scheduled, and by execScheduledActions().
Loop over items, and over formats per item.
"""
# caption = "Executing scheduled actions"
# log = logging.getLogger("execScheduledActions")
h = self.hierarchyWidget
# archivePath, activePath, activeMedium, mediumPattern = archive.getArchiveSettings()
# First collect scheduled items with their formats into a list for progress bar.
result = []
for item in traverseRecursive(h.invisibleRootItem()):
for destFormat in h.formats:
formatId = destFormat.getId()
formatColumn = h.formatColumn[formatId]
actionChar = (getText(item, formatColumn) + " ")[2]
if actionChar != ' ':
result.append((item, destFormat, actionChar))
return result
def execScheduledActions(self):
"""Execute scheduled actions (creating copies etc.).
Loop over items, and over formats per item.
The hierarchyWidget holds with the document hierarchy levels
(series, package, album, side, work, document) also the scheduled actions
for these levels. One action per level and format can be scheduled.
One could imagine to also allow for several different actions
per level and format, such as N and E (create a new copy and export it).
Of the many theoretically conceivable options, we had to first
implement those which are most needed. When deciding what to implement,
it is also important to pay attention to maintain a clear design,
otherwise the program easily gets unmaintainable.
The actions could be executed in several different orders.
One can loop over:
- the items in the hierarchy (i.e., vertically in the table)
- the formats (i.e., horizonally in the table)
- the actionCharSet elements
(for example first all copy actions, then all export actions, etc.)
Currently the following order is implemented:
for item in traverseRecursive(h.invisibleRootItem()):
for format in h.formats:
The action traversal is linearized by:
actionList = self.getScheduledActionList()
and then the actionList is traversed.
"""
caption = "Executing scheduled actions"
log = logging.getLogger("execScheduledActions")
h = self.hierarchyWidget
actionList = self.getScheduledActionList()
steps = len(actionList)
log.info("steps: %d" % steps)
actionCharSet = set([e[2] for e in actionList])
if 'E' in actionCharSet: # for export, select target directory
destDir = archive.selectDir(
"Select target directory for export", "export")
if not destDir: return
multiProgress = progress.MultiProgressDialog(levels=2, windowTitle="Unibas: " + caption,
labelText="<html>%s</html>" % caption)
multiProgress.show()
w0 = multiProgress.progressWidgetList[0]
w0.label.setText("%d actions" % len(actionList))
itemCount = canceled = success = 0
for (item, destFormat, actionChar) in actionList:
w0.progressBar.setValue(100 * itemCount / len(actionList))
# These variables are needed by most actions.
albumItem = h.getAncestorAlbumItem(item)
rn = getNumOr1(item, h.rnColumn)
dn = getNumOr1(item, h.dnColumn)
log.info("Exec %s action on item %s (count: %d) with format %s"
% (actionChar, rn, itemCount, destFormat))
level = getText(item, h.levelColumn)
origDocId = h.getDocId(item)
title = unicode(getText(item, h.dbTitleColumn)
or getText(item, h.extTitleColumn))
filename = "track%02d" % int(dn) \
if h.maintype == "audio" and level == "document" \
else archive.createFilename(title)
destPath = archive.getFreeToarchiveFilepath(
u"{}.{}".format(filename, destFormat.ext()))
albumObjectId = getText(albumItem, h.idColumn)
albumTitle = getText(albumItem, h.dbTitleColumn)
albumDescr = db.descrWithId("object", albumObjectId)
# albumCnt is counter of albums in package, 1-based;
albumCnt = getNumOr1(albumItem, h.rnColumn)
# log.debug("albumObjectId=%s, albumCnt=%d, albumTitle=%s" % (albumObjectId, albumCnt, albumTitle))
log.debug(ds(locals(), "albumObjectId,albumCnt,albumTitle"))
(origDoc, copyDoc) = h.docsToA.getAlbumDocPair(albumCnt, level, rn)
if actionChar == 'R': # ripEncode can be used for testing, leaves no DB traces.
success = h.ripEncode(rn, dn, item, level, origDocId, title,
filename, destPath, destFormat, albumItem,
albumObjectId, albumTitle, albumDescr, albumCnt,
multiProgress)
elif actionChar == 'N':
# success = h.createNewCopy(rn, item, destFormat, albumItem, multiProgress)
success = h.createNewCopy(rn, dn, item, level, origDocId,
title, filename, destPath, destFormat,
albumItem, albumObjectId, albumTitle, albumDescr, albumCnt,
multiProgress)
elif actionChar == 'M':
success = h.createNewRecord(rn, dn, item, albumItem) # TODO: check objectId; better albumItem?
elif actionChar == 'E': # also useful for testing
success = h.exportCopy(rn, dn, item, level, origDocId, title, filename,
destPath, destFormat, albumItem, destDir, multiProgress)
elif actionChar == 'd':
self.dummyAction(rn, item)
else:
msgbox("Action not yet implemented: " + actionChar)
# if progress.wasCanceled():
# break
itemCount += 1
#progress.done(0)
if success:
log.info("Finished.")
if h.dirToImport:
msg = "Delete original directory with contents?\n" + h.dirToImport
if QMessageBox.question(None, caption, msg,
QMessageBox.StandardButtons(QMessageBox.No | QMessageBox.Yes),
QMessageBox.Yes) == QMessageBox.No: return
shutil.rmtree(h.dirToImport)
#if h.discid: # TODO: Reactivate later
#btc("eject " + h.device)
return
log.info("Canceled.")
# View menu ============================================================
def focusContents(self):
print("self.tabWidget.setCurrentWidget(self.contentsPage)")
self.tabWidget.setCurrentWidget(self.contentsPage)
def focusProduct(self):
self.tabWidget.setCurrentWidget(self.productWidget)
def focusObject(self):
self.tabWidget.setCurrentWidget(self.objectWidget)
def focusDocument(self):
self.tabWidget.setCurrentWidget(self.docWidget)
def selectAndAddFormatColumn(self):
h = self.hierarchyWidget
crit = "support=1 AND id not in (%s)" % ','.join(['0'] + [f.getId() for f in h.formats])
if h.maintype: crit += " AND type~'%s/'" % h.maintype
fm = finddialog.FindEntityAndTypeDialog('mimetype')
fm.critEdit.setText(crit)
fm.showCandidates()
if not fm.exec_(): return
id = fm.selectedId()
self.addFormatColumn(id)
def info(self):
msgbox("discid from DB: " + self.productWidget.getValue("discid"))
#def refresh(self):
#objectId = self.objectId
#self.clearContents()
#self.hierarchyWidget.fillFields(objectId=objectId)
# Tools menu ============================================================
def test(self):
caption = "TEST"; h = self.hierarchyWidget
# Working PercentageProcess reading example:
multiProgress = progress.MultiProgressDialog()
multiProgress.show()
btc("rm /tmp/wrap_MP3WRAP.mp3")
command = "mp3wrap /tmp/wrap.mp3 /unibas_archive/oid10645/Fi*.mp3"
p = PercentageProcess(multiProgress, "RipEncode", command,
".* +(\d+) ?%.*", shell=True)
p.startCmd()
print "after startCmd()"
return
# Working DestinationFileProcess example:
multiProgress = progress.MultiProgressDialog()
multiProgress.show()
destPath = '/tmp/cd'; endValue = 700000000
cmd = "dd if=/dev/sr1 of=" + destPath
p = DestinationFileProcess(multiProgress, "RipEncode " + destPath,
cmd, destPath, endValue, shell=True)
p.startCmd()
print "after startCmd()"
return
# Percentage reading example, HandBrakeCLI via ripEncode:
multiProgress = progress.MultiProgressDialog()
multiProgress.show()
h.maintype, h.discid, h.device = "video", '1', "/dev/sr1"
h.objectInDevice = h.objectId = '0'
destFormat = mimetype.Mimetype('133'); destFormat.readFromDb()
h.ripEncode("01", "01", 0, "document", 0, "Dummy", "test.mp4",
"/tmp/test.mp4", destFormat, 0, 0, 0, 0, 0, multiProgress)
print "after ripEncode()"
return
# Reading cdparanoia output fails.
#cmd = "cdparanoia -Z -d /dev/sr1 01 /tmp/track.wav" #- | lame - '/unibas_archive/toarchive/track01.mp3'"
#p = DestinationFileProcess(multiProgress, "RipEncode")
#rs = ".*(\[.*>.*\|).*"
#itemCount = canceled = success = 0
#for (item, format, actionChar) in actionList:
#w0.progressBar.setValue(100 * itemCount / len(actionList))
#albumItem = h.getAncestorAlbumItem(item)
##progress.setValue(itemCount)
##canceled = progress.wasCanceled()
##if canceled: break
#rn = getText(item, h.rnColumn)
#log.info("Exec %s action on item %s (count: %d) with format %s"
#% (actionChar, rn, itemCount, format))
#if actionChar == 'R':
#success = h.ripEncode(rn, item, format, albumItem, multiProgress)
#elif actionChar == 'N':
#success = h.createNewCopy(rn, item, format, albumItem, multiProgress)
#elif actionChar == 'M':
#success = h.createNewRecord(rn, item, albumItem) # TODO: check objectId; better albumItem?
#elif actionChar == 'E':
#success = h.exportCopy(rn, item, format, albumItem, destDir, multiProgress)
#elif actionChar == 'd':
#self.dummyAction(rn, item)
#else:
#msgbox("Action not yet implemented: " + actionChar)
## if progress.wasCanceled():
## break
#itemCount += 1
def ejectAndClear(self):
"""Eject drive and clear."""
h = self.hierarchyWidget
if not h.setDevice(): return
bt("eject " + h.device)
h.clear()
self.productWidget.setId(None)
self.objectWidget.setId(None)
def clearContents(self):
"""Remove all items."""
self.hierarchyWidget.clear()
self.productWidget.setId(None)
self.objectWidget.setId(None)
def dummyAction(self, rn, item):
"""Create new copy of document (e.g. audio track) referenced by item (number: rn), in given format.
"""
caption = "dummyAction"
h = self.hierarchyWidget
btc("sleep 1")
# Commands that produce audio to stdout:
# "mplayer -vo null -vc dummy -af resample=44100 -ao pcm -ao pcm:waveheader:file=/dev/fd/3 3>&1 1>&2 %s" % wmafile
# mplayer src.wma -ao pcm -ao pcm:file=file.wav
# "cdparanoia -d %s %s -" % (device, rn)
# Operations:
# - rip or convert to uncompressed standard format
# - pack or unpack
# - encode in dest. format
# - delete sources
def copyAlbumToArchive(self):
"""Copy album (user selects if there are several) files to archive.
"""
h = self.hierarchyWidget
albumItem = self.getAlbumItem()
origdocidList = []
for item in traverseRecursive(albumItem):
if unicode(item.text(h.levelColumn)) != "document": continue
title = unicode(item.text(h.dbTitleColumn))
origdocid = unicode(item.text(h.idColumn))
print(origdocid, title) #, document.getDigitalVersion(origdocid)
origdocidList.append(origdocid)
document.copyArchivableVersionsToActiveMedium(origdocidList)
def showOwnMediaCounts(self):
"""An albumColumn labeled CD or DVD, MC, LP etc. exists if there are media (original or not)
of one's own, i.e. with roomid NOT NULL. For each such album
(maybe several are displayed in case of packages),
there is an entry in this column that tells the number of own media."""
h = self.hierarchyWidget
for item in traverseRecursive(h.invisibleRootItem()):
if unicode(item.text(h.levelColumn)) != "album": continue
id = unicode(item.text(h.idColumn))
if not id: continue
print(unicode(item.text(h.dbTitleColumn)), ("SELECTED" if item.isSelected() else ''))
def emptyItem(self,item):
"""Remove the children of item."""
for index in range(item.childCount()):
# self.emptyItem(item.child(index))
# item.takeChild(index)
self.emptyItem(item.child(0))
item.takeChild(0)
def setId(self,id):
"""Set ID of object (album).
Clear contents and reread it, starting from product id.
"""
h = self.hierarchyWidget
if id and id == h.objectId: return
# if self.isSubmaster and self.id:
#self.clearContents()
h.clear()
h.objectId = id
print("h.objectId = ", h.objectId)
self.fillFields()
self.emit(SIGNAL("changedId"),id)
def proposedActionLoop(self):
while self.proposeNextAction():
pass
def proposeNextAction(self):
"""Propose next actions and let user decide what to do after fillFields().
Actions can be: impossible, possible but not recommended, possible and recommended:
Import dir if:
- There are dirs to import (type will be decided later)
- cleared, i.e. no h.discid or h.filesToArchive
Download metadata and store in infosourcecontent if:
- no metadata yet in infosourcecontent
- EAN given
- type DVD (h.ot == "DVD"; others later)
Add metadata from infosourcecontent to product and document if:
- EAN given
- metadata for EAN in infosourcecontent
- not yet added, e.g. no cover image for product
Prepare album if:
- source is directory, not (yet) disk: True if h.filesToArchive
- currently only single DVDs: if len(itemList) != 1: return
- metadata have been added
Make copies and remove from schedule if:
- copies are scheduled: self.getScheduledActionList()
- DVD from single dir, and album is prepared, or
- from disc (h.discid)
Eject and clear if:
- disc in drive
- no copies scheduled: self.getScheduledActionList()
Delete dir and clear if:
- no copies scheduled
- for all docs (keys) in h.filesToArchive, there is an origdoc and a copydoc
"""
h = self.hierarchyWidget
archivePath, activePath, activeMedium, mediumPattern = archive.getArchiveSettings()
if h.discid:
originText = "You have checked a disc which can be imported."
elif h.dirToImport:
originText = u"The directory: {}".format(self.dirToImport)
originText += u"has been analyzed and can be imported."
else:
originText = "NIX" # TODO
msg = textwrap.dedent(u"""
<p>{}<br>Please choose one of the proposed actions to do next:</p>
<p><dl>
""")[1:-1].format(originText)
msgBox = QMessageBox()
msgBox.setWindowTitle("Album Form: What to do next?")
itemList = self.getRootDocItemList()
clearButton = None
if itemList:
msg += textwrap.dedent("""
<dt><b>Clear</b></dt>
<dd>Clear display and make ready for next album.</dd>
""")[1:-1]
clearButton = msgBox.addButton("Clear", QMessageBox.ActionRole)
msgBox.setDefaultButton(clearButton)
# Check if importDir is possible and recommended
importDirButton = None
if not h.topLevelItemCount(): # importDir is possible
# Add text to message here.
msg += textwrap.dedent("""
<dt><b>Import directory</b></dt>
<dd>Import album stored in a HD directory.</dd>
""")[1:-1]
importDirButton = msgBox.addButton("Import directory", QMessageBox.ActionRole)
l = importParentDirList
#h.lastImportedType = "DVD"
if h.lastImportedType and h.lastImportedType in l:
l.remove(h.lastImportedType)
l.insert(0, h.lastImportedType)
# print l
for parentDir in l: # TODO: complete list, in settings?
if getSubdirList(archivePath + "raw/" + parentDir):
msgBox.setDefaultButton(importDirButton)
#recommended = True
break
v = self.productWidget.getValue
(productId, ean) = (v("id"), v("ean"))
docId = h.getRootDocId(itemList[0]) if len(itemList) == 1 else None
icList = None; metadataAdded = False
addMetadataButton = downloadMetadataButton = None
if ean:
sql = "SELECT * FROM infosourcecontent WHERE publicid=%s" % q(ean)
print(sql)
icList = db.dicList(sql) # infosourcecontentlist; if icList then metadata are available
print(icList)
if icList: # There are metadata, now check if they are already added.
review = self.docWidget.documentWidget.getValue("review")
if review.strip(): metadataAdded = True
if not metadataAdded: # Recommend adding metadata
msg += textwrap.dedent("""
<dt><b>Add metadata</b></dt>
<dd>Metadata are already downloaded to intermediate tables,
so they can and should be added now.</dd>
""")[1:-1]
addMetadataButton = msgBox.addButton("Add metadata", QMessageBox.ActionRole)
msgBox.setDefaultButton(addMetadataButton)
else: # EAN given and no metadata yet in infosourcecontent
if h.ot == "DVD": # Recommend to Download metadata and store in infosourcecontent if:
msg += textwrap.dedent("""
<dt><b>Download metadata</b></dt>
<dd>Metadata can be downloaded to intermediate tables,
so they can be added later.</dd>
""")[1:-1]
downloadMetadataButton = msgBox.addButton("Download metadata", QMessageBox.ActionRole)
msgBox.setDefaultButton(downloadMetadataButton)
# Prepare album if:
# - source is directory, not (yet) disk: True if h.filesToArchive
# - currently only single DVDs: if len(itemList) != 1: return
# - metadata have been added
prepareAlbumButton = None
# path = h.filesToArchive[docId][0] if docId and h.filesToArchive else ''
path = h.dirToImport or ''
#print("path:", path)
cm = containsMetadata(path)
print("cm:", unicode(cm))
if os.path.exists(path) and len(itemList) == 1:
msg += textwrap.dedent("""
<dt><b>Prepare album</b></dt>
<dd>Metadata can be written to the copy of the album itself,
so it is available without requiring the database.</dd>
""")[1:-1]
prepareAlbumButton = msgBox.addButton("Prepare album", QMessageBox.ActionRole)
if metadataAdded and not cm: msgBox.setDefaultButton(prepareAlbumButton)
# Make copies and remove from schedule if:
# - copies are scheduled: self.getScheduledActionList()
# - DVD from single dir, and album is prepared, or
# - from disc (h.discid)
execActionsButton = None
if self.getScheduledActionList():
msg += textwrap.dedent("""
<dt><b>Execute actions</b></dt>
<dd>Default actions scheduled for new albums are to copy their contents
to files in the default format.</dd>
""")[1:-1]
execActionsButton = msgBox.addButton("Execute actions", QMessageBox.ActionRole)
if cm: msgBox.setDefaultButton(execActionsButton)
##sure if: prod.image, doc.actor
##sure not if: no doc, no ean, no doc.length, no review
# itemList = self.getRootDocItemList()
## if len(itemList) != 1: return
## item = itemList[0]
## docId = self.getRootDocId(item)
#
## docId = self.getRootDocId(itemList[0]) if len(itemList) == 1 else None
#
## length = self.docWidget.documentWidget.getValue("length")
# review = self.docWidget.documentWidget.getValue("review")
# metadataAdded = True if review.strip() else False
#
## DVD album dir is prepared if it contains files, not only *_TS subdirs: getFileList(dir)
## to check if for all docs (keys) in h.filesToArchive, there is an origdoc and a copydoc
# document.getArchivableVersions(origDocId)
msgBox.setText(msg + "</dl></p>")
# keepOrigButton = msgBox.addButton("Keep Original", QMessageBox.ActionRole)
# keepModifButton = msgBox.addButton("Keep Modified File", QMessageBox.ActionRole)
# keepBothButton = msgBox.addButton("Keep Both", QMessageBox.ActionRole)
abortButton = msgBox.addButton(QMessageBox.Abort)
msgBox.exec_()
if msgBox.clickedButton() == abortButton: return 0
elif msgBox.clickedButton() == importDirButton: self.importDirectory()
elif msgBox.clickedButton() == addMetadataButton: self.addMetadata()
elif msgBox.clickedButton() == downloadMetadataButton: self.fetchDataByEan()
elif msgBox.clickedButton() == prepareAlbumButton: self.prepareAlbum()
elif msgBox.clickedButton() == execActionsButton: self.execScheduledActions()
elif msgBox.clickedButton() == clearButton: self.clearContents()
return 1
def showAlbumConnectionTree(self):
h = self.hierarchyWidget
item = h.currentItem() or self.getAlbumItem()
if not item: return
level = getText(item, h.levelColumn)
id = getText(item, h.idColumn)
if id:
entityDic = {"package":"product", "album":"product", "work":"document", "document":"document"}
if not level in entityDic:
errmsg("Unknown entity for level " + level)
return
connectiontree.ConnectionTreeForm(self, entityDic[level], id, 'id').show()
return
id = self.productWidget.getId()
if id and id!='0':
connectiontree.ConnectionTreeForm(self, "product", id, 'id').show()
return
id = self.objectWidget.getId()
if id and id!='0':
connectiontree.ConnectionTreeForm(self, "object", id, 'id').show()
def getFormatIds(self, dir, filelist):
"""Get format IDs from a list of files."""
ids = set()
for path in filelist:
format = mimetype.mimetypeFromFile(dir+'/'+path)
ids.add(format.getId())
return ids
def readDirInfosources():
"""Read directories to be archived under raw/DVD/, raw/CD/ etc.
and fetch metadata for them from infosources,
for those where EANs are provided.
TODO: Prepare this for the case where getPageByEan returns list page, see infosource.getPageByEan
"""
caption = "readDirInfosources"
log = logging.getLogger("readDirInfosources")
archivePath, activePath, activeMedium, mediumPattern = archive.getArchiveSettings()
rawdir = archivePath + "raw/"
il = infosource.infosourceList("type='bookseller'",["re_d_title"])
for otdir in ["DVD"]: #, "CD", "MC", "LP", "book"]:
otpath = rawdir + otdir
if not os.path.isdir(otpath): continue
albumdirList = getSubdirList(otpath) or [] # album dir paths
for dir in albumdirList:
dirOnly = dir.split('/')[-1] # Selected directory name only.
productDic = readProductDir(dirOnly)
if not "ean" in productDic or not productDic["ean"]: continue
ean = productDic["ean"]
#print ean, dirOnly
sql = "select id from infosourcecontent where publicid=%s" % q(ean)
#print sql
#if db.valueList(sql): continue # For this EAN, metadata are already available.
#print "For EAN %s there are no data yet, fetching them." % ean
for i in il:
#print "-------- infosource:", i
msg = "Fetching data for %s from %s %s" % (dirOnly, i.type, i.name)
#print "ID:", i.id
#print msg
log.info(msg)
p = i.getPageByEan(ean)
idList = i.recordDetail(p,'product','ean')
log.info("recordDetail result: " + unicode(idList))
print("recordDetail result: " + unicode(idList))
def readProductDir(dir, src=None):
"""Return dic containing data from directory name.
The directory name can be like: Giving:
ean1234567890123 Godzilla ean="1234567890123" title="Godzilla"
oid12345 Godzilla objectid="12345" title="Godzilla"
pid12345 Godzilla id="12345" (of product) title="Godzilla"
isxn1234567890 Godzilla isxn="1234567890" title="Godzilla"
isbn1234567890 Godzilla isxn="1234567890" title="Godzilla"
Godzilla title="Godzilla"
ICE_AGE_4_CONTINENTAL_DRIFT volumelabel="ICE_AGE_4_CONTINENTAL_DRIFT" title="Ice Age 4 Continental Drift"
"""
dir = dir.split('/')[-1]
m = re.compile("^([0-9X]{13}) (.*)").match(dir)
if m: return {"ean":m.group(1), "title":m.group(2)}
m = re.compile("^ean([0-9X]+) (.*)").match(dir)
if m: return {"ean":m.group(1), "title":m.group(2)}
m = re.compile("^oid([0-9]+) (.*)").match(dir)
if m: return {"objectid":m.group(1), "title":m.group(2)}
m = re.compile("^pid([0-9]+) (.*)").match(dir)
if m: return {"id":m.group(1), "title":m.group(2)}
m = re.compile("^is[xbs]n([0-9]+) (.*)").match(dir)
if m: return {"isxn":m.group(1), "title":m.group(2)}
m = re.compile("^([A-Z0-9_]+)$").match(dir)
if m: return {"volumelabel":m.group(1),
"title":m.group(1).title().replace('_', ' ')}
return {"title":dir}
def readDocFilename(path, defaultTitle="(Unknown)"):
"""Return dic containing data from file name.
The file name can be like: Giving:
track12.cdda file="track12.cdda"
A01 Tomorrow side="A" title="Tomorrow"
A 12:34:56 Tomorrow file="A 12:34:56" title="Tomorrow"
Godzilla title="Godzilla"
01 First chapter title="First chapter" (01 is ignored, used for order only)
"""
filename = path.split('/')[-1] #.replace('_', ' ')
m = re.compile("^(track\d\d.cdda)").match(filename)
if m: return {"file":m.group(1)}
m = re.compile("^([AB]?)\d\d? ?(.*)").match(filename)
if m: return {"side":m.group(1), "title":archive.fileToTitle(m.group(2), defaultTitle)}
m = re.compile("^([AB] \d\d:\d\d:\d\d) ?(.*)").match(filename)
if m: return {"file":m.group(1), "title":archive.fileToTitle(m.group(2), defaultTitle)}
return {"title":archive.fileToTitle(filename)}
def isDvdStructure(dir, parentDir=None):
"""Return True if dir is a DVD directory structure, containing VIDEO_TS.
If parentDir is given, also check that is "DVD".
"""
if parentDir and parentDir != "DVD": return False
subdirList = getSubdirList(dir) or []
if ("VIDEO_TS" in subdirList or "video_ts" in subdirList):
return True
return False
def selectDir(lastImportedType):
caption = "Select directory to import"
# startdir = settings.getSetting("AlbumForm/importBaseDir")
# if not startdir:
archivePath, activePath, activeMedium, mediumPattern = archive.getArchiveSettings()
startdir = archivePath + "raw/"
if lastImportedType and lastImportedType in importParentDirList:
startdir += lastImportedType + '/'
subdirList = getSubdirList(startdir)
if subdirList: startdir += subdirList[0] + '/'
dialog = QFileDialog(None, caption, startdir.encode("utf-8"))
dialog.setFileMode(QFileDialog.Directory)
if not dialog.exec_(): return None
dir = unicode(dialog.selectedFiles()[0])
# settings.setSetting("AlbumForm/importBaseDir", os.path.dirname(dir))
return dir
def selectFile(lastImportedType):
caption = "Select file to import"
archivePath, activePath, activeMedium, mediumPattern = archive.getArchiveSettings()
startdir = archivePath + "raw/"
if lastImportedType and lastImportedType in importParentDirList:
startdir += lastImportedType + '/'
#subdirList = getSubdirList(startdir)
#if subdirList: startdir += subdirList[0] + '/'
dialog = QFileDialog(None, caption, startdir.encode("utf-8"))
dialog.setFileMode(QFileDialog.ExistingFile)
if not dialog.exec_(): return None
selectedFile = unicode(dialog.selectedFiles()[0])
return selectedFile
def getProductidFromObjectidInDic(product):
if not "objectid" in product.vd: return None
dl = db.dicList("SELECT productid FROM object WHERE id=" + product.vd["objectid"])
if not dl:
msg = "Tried to get product ID from nonexisting objectid %s" % product.vd["objectid"]
errmsg(msg)
return None
return dl[0]["productid"]
def askAndWrapAlbum(ot, albumDir, productDirOnly, fileList):
"""If the album content is best represented as one document,
ask and concatenate files to one.
In this case return True, else False.
TODO: Do this later, as scheduled action.
"""
caption = "Checking if %s is one document" % productDirOnly
if ot not in ("MC", "LP"): return # One file per side. TODO: Later also include CDs.
exts = list(set([os.path.splitext(f)[1] for f in fileList])) # Including dots, e.g. set(['.mp3', '.jpg'])
if len(exts)>2: return 0 # Not what we are looking for.
audioExtList = [".mp3",".ogg",".wmv"] # TODO: make configurable.
imageExtList = [".jpg"] # TODO: make configurable.
# One ext must be of an audio type, the other, if existing, an image type.
audioExtIdx = 0 # Suppose the audio ext is the first, i.e. its index is 0.
if len(exts) == 1:
if exts[0] not in audioExtList: return 0
else: # Two exts. Find out which (index) is audio type.
if exts[0] not in audioExtList: audioExtIdx = 1
if exts[audioExtIdx] not in audioExtList: return 0
if exts[1-audioExtIdx] not in imageExtList: return 0
audioExt = exts[audioExtIdx]
audioFileList = [f for f in fileList if f.endswith(audioExt)]
print("audioFileList", audioFileList)
if len(audioFileList) != 2: return 0 # Exactly one file per side.
msg = textwrap.dedent("""
This directory: %s
seems to hold audio files: %s
that are best archived as one document.
Concatenate them and archive them as one document?
""")[1:-1] % (productDirOnly, ','.join(audioFileList))
if QMessageBox.question(None, caption, msg,
QMessageBox.StandardButtons(QMessageBox.No | QMessageBox.Yes),
QMessageBox.Yes) == QMessageBox.No: return 0
# Better use mp3wrap or so?
cmd = "cat '%s/%s' >>'%s/%s'" % (albumDir, audioFileList[1], albumDir, audioFileList[0])
print(cmd)
btc(cmd)
cmd = "rm '%s/%s'" % (albumDir, audioFileList[1])
print(cmd)
btc(cmd)
return 1
#now in product
def askAndRecordProductImage(ot, albumDir, productId):
"""If main directory contains image files (esp. jpg),
then for each image ask if it shows front cover, back cover or product (MC/disk) itself.
If so, register it as document and connect it with rel to product.
See document.downloadImage
"""
caption = "Checking product images"
#print ot, albumDir
fileList = getFileList(albumDir)
allExts = set([os.path.splitext(f)[1] for f in fileList]) # Including dots, e.g. set(['.mp3', '.jpg'])
supportedImageExtsList = [".jpg"] # TODO: make configurable.
imageExtSet = allExts.intersection(set(supportedImageExtsList))
#print "imageExtSet ", imageExtSet
if len(imageExtSet) != 1: return 0
imageExt = list(imageExtSet)[0]
imageFileList = [f for f in fileList if f.endswith(imageExt)]
print("imageFileList ", imageFileList)
# If user clicks Cancel on one of the images, the function aborts.
# If user clicks Skip on one of the images, only that image is skipped.
fileAnswers = {}
for (j0, file) in enumerate(imageFileList): # Loop through files sorted alphabetically.
(base,ext) = os.path.splitext(file)
j = j0 + 1
path = albumDir + '/' + file # Complete (source) file path.
print(path)
im = AlbumImageDialog(path)
if base == 'front': im.frontCoverButton.setFocus()
elif base == 'back': im.backCoverButton.setFocus()
if not im.exec_(): return # User clicked "Cancel".
fileAnswers[file] = im.answer
ta = archive.FileDbTransaction("Archiving cover and product image files")
for (file, answer) in fileAnswers.items():
if not answer: continue # Case when user clicked "Skip".
#print "reg. image file: %s, relid:%s" % (file, answer)
path = albumDir + '/' + file
# Archive this file as document and rel it to product.
doc = document.Document(src=path)
dupid = doc.checkDuplicate()
if dupid:
print("%s already known as doc %s" % (file, dupid))
continue
attributes = dict() # Will be filled by metadatareader.
metadatareader.readFromFile(attributes, path)
metadatareader.selectMaxLikelihood(attributes)
doc.getMetadataValues(attributes)
id = document.createDocumentRecord(doc, caption)
relDic = {"ent1":"5", "id1":id, "relid":answer, "ent2":"23", "id2":productId}
#print "relDic:", relDic
db.insertDic("rel", relDic)
def containsMetadata(path):
"""Return True if path is album (file or directory; not package) that contains metadata.
If not or in case of doubt, return False.
"""
# print "containsMetadata: path", path
if not path: return None
if os.path.isdir(path):
# print "containsMetadata: isdir"
if os.path.isfile(path + '/VIDEO_TS/VIDEO_TS.IFO'):
# print "isfile", path + '/VIDEO_TS/VIDEO_TS.IFO'
l = getFileList(path)
# print l
if l: return True
return False
#def getLevRatio(albumList):
def getLevRatio(docsToA):
"""Return minimum Levenshtein rate."""
levRatio = 1.0
oldTitle = None
# for album in albumList:
for i, album in docsToA.orig.iteritems():
for origDoc in album:
t = origDoc.vd["title"] if origDoc else ''
if oldTitle:
levRatio = min(levRatio, Levenshtein.ratio(oldTitle, t))
oldTitle = t
#print "oldTitle: %s, levRatio: %4.2f" % (oldTitle, levRatio)
return levRatio
def getItemDescr(h, item):
level = getText(item, h.levelColumn)
id = getText(item, h.idColumn) or "(no ID)"
file = getText(item, h.fileColumn) or "(no file)"
title = getText(item, h.dbTitleColumn) or "(no DB title)"
extTitle = "(ext. title: %s)" % (getText(item, h.extTitleColumn) or "none")
rawCnt = getText(item, h.rnColumn)
cnt = "#%s " % rawCnt if rawCnt else ''
# "document 12345: #01 Title (ext. title: none) [file: (no file)]
return "%s %s: %s%s %s [file: %s]" % (level, id, cnt, title, extTitle, file)
def getDescendantItems(h, topItem, level=None):
"""Return list of items below given topItem,
filtered by level if given."""
result = []
for item in traverseRecursive(topItem):
if level and getText(item, h.levelColumn)!=level: continue
result.append(item)
return result
#def getAncestorAlbumId(h, item):
#"""Return ID of given item if it is an album or of
#an ancestor if that is an album.
#"""
#while 1:
#if getText(item, h.levelColumn) == "album":
#return getText(item, h.idColumn)
#item = item.parent()
#if not item: return None
# print type(item)
# return None
#def getAncestorAlbumItem(h, item):
#"""Return given item if it is an album or
#the ancestor that is an album.
#"""
#while 1:
#if getText(item, h.levelColumn) == "album": return item
#item = item.parent()
#if not item: return None
def getProductRootDocId(productId):
sql = textwrap.dedent("""
SELECT d.id FROM product p, object o, cp, document d
WHERE o.productid=p.id AND cp.objectid=o.id AND cp.file='/'
AND cp.documentid=d.id and p.id=%s;
""") % productId
vl = db.valueList(sql)
return vl[0] if vl else None
#def getObjectProductList(objectId, productId):
#"""Get a list of tuples (cnt, objectId, productId)
#where the tuple with index 0 is the package (may be (0, None, None))
#and the following tuples represent albums.
#Input may be an album (object and/or product)
#or package product. At least one of objectId, productId must be not None.
#However, for each product only one album is returned.
#That album is objectId if it is given, otherwise an original, if there is.
#The list is ordered by object.descr.
#"""
#log = logging.getLogger("getObjectProductList")
#log.debug("objectId: %s, productId: %s" % (objectId, productId))
## Decide if it is a package or album.
## - no product ID given and none in DB -> album
## - product ID given and has prods -> package
## - product ID given and has packageId -> album
#if not productId:
#productId = db.readSql("SELECT productid FROM object WHERE id=%s" % objectId, 0, 1)
#sqlBase = "SELECT id FROM product p WHERE p.packageid=%s"
#prods = db.valueList(sqlBase % productId) if productId else None
#log.debug("prods: %s" % unicode(prods))
#if not prods: # Product is not package but still may be part of a package.
#if not productId: return [(0, None, None), (1, objectId, productId)]
#packageId = db.readSql("SELECT packageid FROM product WHERE id=%s" % productId, 0, 1)
#if not packageId: return [(0, None, None), (1, objectId, productId)] # Definitely no package.
#prods = db.valueList(sqlBase % packageId) # is part of package, now select package's parts
#packageObjectId = db.readSql("SELECT id FROM object WHERE productid=%s" % packageId, 0, 1)
#res = [(0, packageObjectId, packageId)]
#else: # productId is package
#res = [(0, objectId, productId)]
#objectList = []
#cnt = 0
#for productId in prods: # loop over package parts
#sql = "SELECT id, descr FROM object WHERE productid=%s" % productId
#sql += " ORDER BY (id=%s) DESC, orig DESC;" % (objectId or 0)
## ordering should ensure that given objectId is first, otherwise orig is first
#dl = db.dicList(sql)
#if dl: objectList.append((productId, dl[0]["id"], dl[0]["descr"]))
##print objectList
##print 90*'-'
## Sort objectList by descr which has index 2 in the tuples.
#objectList.sort(key=lambda x: x[2])
##print objectList
##print 90*'='
#for o in objectList:
#cnt += 1
## res.append((cnt, vl[0], productId))
#res.append((cnt, o[1], o[0]))
#return res
#def getObjectProductList(objectId, productId):
#"""Get a list of tuples (cnt, objectId, productId)
#where the tuple with index 0 is the package (may be (None, None))
#and the following tuples are albums.
#Input may be an album or package. At least one of objectId, productId must be not None.
#Ordering is by album, i.e. object.
#However, for each product only one album should be shown.
#That album should be objectId if it is given, otherwise an original, if there is.
#The list then must be ordered by object.descr.
#"""
#log = logging.getLogger("getObjectProductList")
#log.debug("objectId: %s, productId: %s" % (objectId, productId))
## Decide if it is a package or album.
## - no product ID given and none in DB -> album
## - product ID given and has prods -> package
## - product ID given and has packageId -> album
#if not productId:
#productId = db.readSql("SELECT productid FROM object WHERE id=%s" % objectId, 0, 1)
#sqlBase = textwrap.dedent("""
#SELECT o.id as oid, p.id as pid, o.descr FROM product p, object o
#WHERE p.packageid=%s AND o.productid=p.id ORDER BY convert_to(o.descr, 'utf8');
#""") # See below for ordering.
#prods = db.dicList(sqlBase % productId) if productId else None
#log.debug("prods: %s" % unicode(prods))
#if not prods: # Product is not package but still may be part of a package.
#if not productId: return [(0, None, None), (1, objectId, productId)]
#packageId = db.readSql("SELECT packageid FROM product WHERE id=%s" % productId, 0, 1)
#if not packageId: return [(0, None, None), (1, objectId, productId)] # Definitely no package.
#prods = db.dicList(sqlBase % packageId) # is part of package, now select package's parts
#packageObjectId = db.readSql("SELECT id FROM object WHERE productid=%s" % packageId, 0, 1)
#res = [(0, packageObjectId, packageId)]
#else: # productId is package
#res = [(0, objectId, productId)]
#for (cnt, product) in enumerate(prods): # loop over package parts
#res.append((cnt+1, product["oid"], product["pid"]))
#return res
def sumDigits(chk, start=0, end=1, step=2, mult=1):
return reduce(lambda x, y: int(x)+int(y), list(chk[start:end:step])) * mult
# http://www.postgresql.org/docs/current/static/isn.html
def eanCheckDigit(ean):
"""Return the checksum digit of an EAN-13/8 code.
TODO
"""
l = len(ean)
m0 = 2*(l%2)+1
m1 = 4-m0
print(m0, m1)
#if chk.isdigit() and __lenCheck(chk):
#if code == 'EAN13':
#m0=1
#m1=3
#elif code == 'EAN8':
#m0=3
#m1=1
#else:
#return None
# _len = codeLength[code]-1
t = 10 - (( sumDigits(chk, start=0, end=l-1, mult=m0) + \
sumDigits(chk, start=1, end=l-1, mult=m1) ) %10 ) %10
return t if t!=10 else 0
#if t == 10:
#return 0
#else:
#return t
#return None
# To include spaces etc. in ordering, order by convert_to(descr, 'utf8'),
# not by descr itself. Tip from:
# http://stackoverflow.com/questions/4955386/postgresql-ignores-dashes-when-ordering
# This is strange because charset is UTF8 already:
# # show lc_collate;
#lc_collate
#-------------
#en_US.UTF-8
# select descr from object where id>13659 order by descr;
# select descr from object where id>13659 order by convert_to(descr, 'utf8');
# Note that album objects may be in a certain order,
# but that order can be found nowhere in the data.
# It is simply the order of the discs in the package.
# Example: "Planet Erde" BBC documentation DVD package EAN 4006448756352
# which contains 6 DVDs the last one of which is named "Bonus Disc"
# and would come first by alphabetic ordering, but
# has the last place in the package, and by the
# convention of placing bonus discs last.
# In this case it may be useful to invent album names as follows:
# "Packagetitle DVD 1" .. "Packagetitle DVD 6"
# maybe extended by the names written on the discs:
# TODOs:
# After adding new role, take last role as default and select the new role
# Show (playback) parts if the whole is requested but not available, and vice versa
# Error message if Show external version and the wrong CD is in the drive
# Make sure MP3 files of whole albums can be split up in parts
# and part titles are conserved.
# Default-select "Add new"; automatically return new entity after AddNew
# For roles:
# after creating new role, select it
# provide default ord (last ord +1)
# provide default language (from document or from person)
# in FindDialog, don't require '+' for providing search string as default
# For persons, guess sex
"""
"Artist" is a generic term, encompasses pop singers,
classical voice types, etc.; it is suitable in role table,
but if more accurate information is available,
e.g. voice type, that should be used.
For (classical) voice type, the notion of that voice type
should be used, not the notion of the person with that type.
Note that classical opera singers can well sing in
(for them) foreign languages, e.g. German singers can sing
roles in Italian etc.; guessing a role's language from the
document is more accurate than from the person's language.
Where a person or a company needs to be provided,
both fields (personid and companyid) are present.
Examples are owner and role tables, but more such cases
may occur in the future. Both fields may be filled, then
this means "company ... under the direction of person ...".
12858 | chorus | a group of people assembled to sing together | 6890
2 | composer | person who writes (composes) music | 6880
58012 | tenor voice | the adult male singing voice above baritone | 6862
63811 | artist | someone who practises one of the fine arts, e.g. painter, or performs music vocally or instrumentally, etc. | 6861
63813 | orchestra | organized group of musicians | 6860
63886 | soprano | the highest female voice | 6859
39459 | mezzo-soprano | the female singing voice between contralto and soprano | 6834
58012 | tenor voice | the adult male singing voice above baritone | 6836
5939 | baritone voice | the second lowest adult male singing voice | 6850
6153 | basso | the lowest adult male singing voice | 6839
The role.ord field roughly gives the order of importance.
Note that operas on CDs are rarely complete, only highlights
are recorded. The overture, arias etc. of such an opera on CD
constitute a work of their own, which may be seen as a version
of the original opera composition. It thus inherits properties
from that composition (such as the composer, language etc.),
although it is not complete, i.e. not all parts of the composition
are represented in this work.
Take doc 77572: Fidelio as an example of such a work.
Here all roles are focused in the work document.
Another example is doc 77522: The Magic Flute, where
the single arias etc. are provided with roles information,
which is more accurate.
This information, together with some general text about the
composition (opera) and maybe the composer,
should replace a CD's supplement and back cover completely.
The goal of Unibas is not only to archive a CD completely,
but also the information on its packaging, supplement etc.
UnDVD handling:
dvdbackup
try genisoimage -gui -dvd-video -o "%s" "%s"' % (dest, src)
if successful:
archive result .iso file
else:
cat VTS_nn_i.VOB
with archive.catUndvd(undvd)
"""
class UnibasProcess():
"""A lengthy process (e.g. file copying)
the progress of which needs to be shown.
"""
def __init__(self, progressDialog, labelText):
self.progressDialog = progressDialog
self.labelText = labelText
# e = multiProgress.startFunction(cmd, dest) #"uhash('%s', widget)" % (archive.fullPath(objectid, file).replace("'", "\\'")), file)
class DestinationFileProcess(UnibasProcess):
"""A process that creates a file of (approximately) known size.
The command is executed, it is expected to create the file sizeFile.
That file is expected to grow up to endValue.
So the completion percentage is calculated as:
100 * size / endValue
where size is the actual size of the sizeFile.
Stdout is not read, but maybe stderr for error messages.
"""
def __init__(self, progressDialog, labelText, cmd, destPath, endValue, shell=False):
UnibasProcess.__init__(self, progressDialog, labelText)
self.command, self.destPath, self.endValue, self.shell = \
cmd, destPath, endValue, shell
def startCmd(self):#, command, sizeFile, endValue, shell=False):
"""Start a command and watch progress as file size."""
log = logging.getLogger("startCmd")
if log.isEnabledFor(logging.DEBUG):
if isinstance(self.command, list):
log.debug("command: %s" , ' '.join(self.command))
else:
log.debug("command: %s" , self.command)
self.process = subprocess.Popen(self.command, shell=self.shell)
# QCoreApplication.instance().processEvents() # get app for remaining responsive
progressDialog = self.progressDialog
progressDialog.setBottomWidgetLabelText(self.labelText)
progressDialog.setBottomWidgetProcess(self.process)
progressDialog.setBottomWidgetValue(0)
self.abort = False
while self.process.poll() is None:
QCoreApplication.instance().processEvents() # get app for remaining responsive
size = 0
if os.path.isdir(self.destPath):
for root, dirs, files in os.walk(self.destPath):
for file in files:
size += os.path.getsize(root + '/' + file)
elif os.path.isfile(self.destPath):
size = os.path.getsize(self.destPath)
if size > 0:
progressDialog.setBottomWidgetValue(100 * size / self.endValue)
time.sleep(1)
log.info("process no longer running")
progressDialog.setBottomWidgetValue(100)
self.exitCode = self.process.returncode
self.success = True
res = ''
if self.abort or self.exitCode:
self.success = False
res = "Process exited abnormally.\n"
if self.abort:
res += "Reason: Abort by user\n"
else:
res += "Exit code: " + unicode(self.exitCode) + "\n"
if self.exitCode and not self.abort:
log.error(res)
return res
class PercentageProcess(UnibasProcess):
"""A process that reads its progress information from stdout or stderr.
By default the percentage is read from stdout.
A default regexp is provided for this, however, it is recommended
to provide a regexp tailored for the situation.
Tested with HandBrakeCLI
TODO: Read stderr as well? What to do with it,
detect errors or remember it for later?
"""
def __init__(self, progressDialog, labelText, command,
outRegExpString="[^\d](\d+) ?%",
shell=False, bufsize=0):
UnibasProcess.__init__(self, progressDialog, labelText)
self.command, self.shell, self.bufsize = \
command, shell, bufsize
self.outRegExp = re.compile(outRegExpString) if outRegExpString else None
def startCmd(self):
"""Start a command and watch progress."""
log = logging.getLogger("startCmd")
if log.isEnabledFor(logging.DEBUG):
if isinstance(self.command, list):
log.debug("command: %s" , ' '.join(self.command))
else:
log.debug("command: %s" , self.command)
self.process = subprocess.Popen(self.command,
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
shell=self.shell, bufsize=self.bufsize)
progressDialog = self.progressDialog
progressDialog.setBottomWidgetLabelText(self.labelText)
progressDialog.setBottomWidgetProcess(self.process)
progressDialog.setBottomWidgetValue(0)
self.abort = False
while self.process.poll() is None:
QCoreApplication.instance().processEvents() # get app for remaining responsive
self.readFromStdout()
# time.sleep(1)
log.info("process no longer running")
progressDialog.setBottomWidgetValue(100)
self.exitCode = self.process.returncode
self.success = True
res = ''
if self.abort or self.exitCode:
self.success = False
res = "Process exited abnormally.\n"
if self.abort:
res += "Reason: Abort by user\n"
else:
res += "Exit code: " + unicode(self.exitCode) + "\n"
if self.exitCode and not self.abort:
log.error(res)
return res
def readFromStdout(self):
# s = uc(self.process.readAllStandardOutput())
#print "before read()"
s = self.process.stdout.read(80)
ps = ''.join(c for c in s if ord(c)>=32)
#log.debug("From stdout: " + ps)
print "From stdout: " + ps
if self.outRegExp:
m = self.outRegExp.match(s) #,0)
if m:
print "matched"
# rawProgressValue = eval(self.extractExpr) if self.extractExpr else m.group(1)
rawProgressValue = m.group(1)
print "rawProgressValue ",rawProgressValue
#print "scaleValue: ", self.scaleValue
self.progressDialog.setBottomWidgetValue(int(rawProgressValue))
# self.progress.setValue(int(float(rawProgressValue) * self.scaleValue))
return
# if self.rememberStdout: self.outOutput += s
