[Crash] "Database needs recovery"

[mrcnski]

Hello friend, and many thank you for bestowing this command line creation upon humanity.

I beseech your expert help in a capacity that, I hope, far exceeds my own abilities.

My good friend, when I hath invoketh proselint on any file of textual contents, the following is what I invariably witness:

Traceback (most recent call last):
  File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shelve.py", line 111, in __getitem__
    value = self.cache[key]
KeyError: 'd7506be8fd0043d2cbf09070321c5ca97ce42133919bfd15d0801cb8df1c3c64'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/proselint/0.10.2_1/libexec/lib/python3.7/site-packages/proselint/tools.py", line 134, in wrapped
    return cache[key]
  File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shelve.py", line 113, in __getitem__
    f = BytesIO(self.dict[key.encode(self.keyencoding)])
KeyError: b'd7506be8fd0043d2cbf09070321c5ca97ce42133919bfd15d0801cb8df1c3c64'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/proselint/0.10.2_1/libexec/lib/python3.7/site-packages/proselint/command_line.py", line 139, in proselint
    errors = lint(f, debug=debug)
  File "/usr/local/Cellar/proselint/0.10.2_1/libexec/lib/python3.7/site-packages/proselint/tools.py", line 257, in lint
    result = check(text)
  File "/usr/local/Cellar/proselint/0.10.2_1/libexec/lib/python3.7/site-packages/proselint/tools.py", line 137, in wrapped
    cache[key] = value
  File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shelve.py", line 125, in __setitem__
    self.dict[key.encode(self.keyencoding)] = f.getvalue()
_gdbm.error: Database needs recovery

What could this mean? I am at a loss, for long have I pondered, but even Google does not knows nothing.

$ proselint --version
0.10.2

I use OSX, Emacs, and Firefox.

[mrcnski]

After trying to reinstall it (no instructions on how to do this in the Readme, btw...) I see the following error instead:

~/repos/QA/Documentation $ proselint Add\ New\ Repository.md 
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/proselint/command_line.py", line 139, in proselint
    errors = lint(f, debug=debug)
  File "/usr/local/lib/python2.7/site-packages/proselint/tools.py", line 257, in lint
    result = check(text)
  File "/usr/local/lib/python2.7/site-packages/proselint/tools.py", line 137, in wrapped
    cache[key] = value
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shelve.py", line 133, in __setitem__
    self.dict[key] = f.getvalue()
error: Database needs recovery

[neonfuzz]

I ran into the same error and did a handful of things; one of them worked (though I'm not sure which).

• Uninstalled proselint
• I removed all of the proselint.vim files from my ~/.vim/bundle/ale directory; I doubt this was the issue, but I'm including it for completeness.
• I removed the directory ~/.cache/proselint (Strong hunch this was the culprit)
• Re-installed proselint (using pip fwiw)
• Ran proselint with the -c option (which ostensibly clears the cache, possibly the same as removing the ~/.cache/proselint directory)

And then it worked! I wish I had done a little better science on it, but hopefully one of those steps will work for you as well.

[mrcnski]

Thanks! I'm sure that will be helpful to some, but I decided not to continue using an abandoned project. Also, I noticed some weird stuff in the readme:

lgbtq.offensive_terms | Avoding offensive LGBTQ terms
lgbtq.terms | Misused LGBTQ terms

Like, wtf? I just want a non-political grammar checker that doesn't blast me with radical leftist bullshit!

[Nytelife26]

It seems that this has been long-resolved, so it will be closed. Although, as an update, proselint is now re-entering active maintenance.

I'd appreciate it if someone mentioned me here if anyone still suffers from this issue, and I can re-open it and investigate properly.

Although...

Like, wtf? I just want a non-political grammar checker that doesn't blast me with radical leftist bullshit!

This isn't a political statement, nor "bullshit", as you kindly term it. Instead, proselint is, well, a linter for prose. Part of writing proficiency is ensuring you convey the meaning you intend and can get your point across without rooting it in harmful ideology.

After all, linguistics evolves, and contexts change.

[vEnhance]

I'd appreciate it if someone mentioned me here if anyone still suffers from this issue, and I can re-open it and investigate properly.

Since you asked --- I'm still getting this issue, but can't reproduce it. Basically, seems like once every 3-5 weeks, I'll run into this error. Deleting ~/.cache/proselint/ always fixes it, so it's only a minor annoyance. Not sure it's worth your time to deal with, but figured to mention it anyways. (I don't have a full traceback right now, sadly.)

I'm using Arch Linux and Vim, have proselint being called through ALE. Proselint is version 0.10.2.

[Nytelife26]

Since you asked --- I'm still getting this issue, but can't reproduce it.

That's weird. We will have to determine an exact cause and way to force reproduction of the issue before it can be triaged successfully, but thank you for informing me someone is still having to deal with it. I'll look into it.

I'm using Arch Linux and Vim, have proselint being called through ALE. Proselint is version 0.10.2.

Unrelated, but I'll commend your choice of environment. I'm using the same setup, actually.

[Nytelife26]

Having looked around online, it seems that the cause of issues like this is that the cache is unable to write properly once it reaches a certain size, and thus keys fail to index and the database gets corrupted. That also aligns with @vEnhance's report that it occurs every 3-5 weeks, since the database will grow over time and perhaps that is when it grows too large under their circumstances.

I'll test this now by forcibly running many different things through the lint cache and seeing if it eventually breaks when it gets large.

[Nytelife26]

It seems that the best solution would be to hard wipe the cache in a similar fashion to the --debug CLI flag if memoization fails. That, or we could use heatmap-based caching, where only inputs to each function that are used frequently are stored.