Not logged in
Ticket UUID: 237f9e4d172a5eba3e39387b31c862820d0b5da6
Title: Cannot add "Lord Bellwater's Secret" to the Grotesque db
Status: Open Type: Code_Defect
Severity: Severe Priority: Immediate
Subsystem: Resolution: Open
Last Modified: 2015-10-13 21:55:17
Version Found In: 0.9.5
User Comments:
anonymous added on 2015-10-04 10:03:01:
(I'd like to test from the code from HEAD but I cannot see a way to download it other than on file-by-file basis.)
Adding the game http://ifdb.tads.org/viewgame?id=pt3qanr6p6mzauho to the Grotesque (0.9.5), using pyiflabel (0.4) on Ubuntu 15.04 - results in the following stacktrace:

Traceback (most recent call last):
  File "/home/pwiecz/lib/grotesque/gui/gtk3/mainwindow.py", line 967, in on_close
    self.save_library()
  File "/home/pwiecz/lib/grotesque/gui/gtk3/mainwindow.py", line 368, in save_library
    self.library.save(lib_handle, GROTESQUE_VERSION)
  File "/home/pwiecz/lib/grotesque/gui/gtk3/library/library.py", line 512, in save
    data['library'] = self.serialize_liststore()
  File "/home/pwiecz/lib/grotesque/gui/gtk3/library/library.py", line 404, in serialize_liststore
    row[15] = [story.to_ifiction(indent=""), story.cover.data,
  File "/home/pwiecz/lib/python2.7/site-packages/treatyofbabel/ifstory.py", line 244, in to_ifiction
    return doc.toprettyxml(indent=indent, encoding="UTF-8")
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 58, in toprettyxml
    self.writexml(writer, "", indent, newl, encoding)
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 1749, in writexml
    node.writexml(writer, indent, addindent, newl)
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 814, in writexml
    node.writexml(writer, indent+addindent, addindent, newl)
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 814, in writexml
    node.writexml(writer, indent+addindent, addindent, newl)
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 814, in writexml
    node.writexml(writer, indent+addindent, addindent, newl)
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 814, in writexml
    node.writexml(writer, indent+addindent, addindent, newl)
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 810, in writexml
    self.childNodes[0].writexml(writer, '', '', '')
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 1038, in writexml
    _write_data(writer, "%s%s%s" % (indent, self.data, newl))
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 298, in _write_data
    writer.write(data)
  File "/usr/lib/python2.7/codecs.py", line 357, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 64: ordinal not in range(128)

anonymous added on 2015-10-04 10:22:48:
Apparently the stacktrace is independent from the error.
The stacktrace starts showing up once I've added e.g. "Earth and Sky 2" - http://ifdb.tads.org/viewgame?id=toa3u84xtn0211en

Adding Earth and Sky 2 also makes the Grotesque unusable - it cannot save the db, and hangs while exiting.

brandon added on 2015-10-04 11:29:01:

Hi,

I'm sorry that this bug is rendering Grotesque unusable for you. I've put in a lot of work since 0.9.5 to make the program more robust and stable. It's not quite ready for release yet but it's close.

I've tried adding both of the games you listed to my library with the latest version and they both work well (EaS coverart doesn't load but that's a separate problem; it doesn't appear to be png or jpeg). Can you please test it out?

To download the latest version, use:

Latest Grotesque commit

This requires the lastest version of pyifbabel:

Latest pyifbabel commit

(for future reference: in the "Dev Log" section of this page, you can click on any commit's link and be taken to a page with links to download .tar.gz or .zip archives of the commit)

The latest version uses a completely different underlying database format, so it will not overwrite your old database. If you decide to continue to use the development version, I recommend making periodic backups of the file $HOME/.local/share/grotesque/local_library.sqlite. The database format will remain stable for the foreseeable future and I'm pretty sure that I've squashed all the bugs that might affect it, but I'm still testing it.


anonymous added on 2015-10-13 21:55:17:
It seems to be much more usable when build from the sources you've provided - thanks!

It gets stuck on downloading metadata or coverart from ifdb pretty often, but interrupting it, doesn't seem to cause any problem.

When it does it prints the stacktrace:

Traceback (most recent call last):
  File "/home/pwiecz/lib/python2.7/site-packages/grotesque/gui/gtk3/threads/storyimportthread.py", line 110, in import_stories
    story_id, ifid, ifformat, os.path.realpath(filename))
  File "/home/pwiecz/lib/python2.7/site-packages/grotesque/gui/gtk3/threads/storyimportthread.py", line 227, in _add_story_release
    format_id)
  File "/home/pwiecz/lib/python2.7/site-packages/grotesque/db/__init__.py", line 527, in insert_release
    compiler_version, format_id))
sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

It does so for e.g. http://ifdb.tads.org/viewgame?id=toa3u84xtn0211en - maybe that's the problem you've mentioned.

BtW, I've had to modify setup.py and add 'grotesque.db' to the list of packages, to make it work.