New release of better bibtext breaks zotxt (BetterBibTeX.KeyManager.keys.findOne is not a function)

[newhallroad]

I believe a new release of better bibtext has broken something in zotxt. The key issue seems it might be TypeError: zotero.BetterBibTeX.KeyManager.keys.findOne is not a function

My setup:
Zotero 6.0.27
Better BibTex for Zotero 6.7.132
zotxt 6.0.2
I am accessing zotxt through the pandoc-zotxt.lua filter in emacs 29.1 on Windows 10

Zotxt failed to find items in my Zotero bibliography. When I rolled back to Better Bibtext v6.7.100, things worked as they should. (I chose that version arbitrarily; I didn't test every release going back.)

The Zotero debugger gives me the output below. I have included the entries on either side of the errors.

This is wonderful project. I am very grateful to you. I hope you will be kind enough to fix this issue.

(3)(+0005003): {better-bibtex} +5002 idle: Thu Oct 26 2023 23:46:04 GMT-0400 (Eastern Daylight Time), save-database active

(3)(+0000002): {better-bibtex} +2 idle: auto-export: {"state":{"state":"active","topic":"save-database"},"pref":{"autoExport":"immediate","autoExportIdleWait":10}}

(5)(+0011210): GET /zotxt/items?betterbibtexkey=sagiv_DeepBlueNotes_2015 HTTP/1.1 Host: localhost:23119 Accept-Encoding: gzip

(3)(+0000008): TypeError: zotero.BetterBibTeX.KeyManager.keys.findOne is not a function findByBBTKey@chrome://zotxt/content/modules/Core.jsm:217:18 itemsEndpoint/<@resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///C:/Users/alexanderkaye/AppData/Roaming/Zotero/Zotero/Profiles/79uiu4p8.default/extensions/[email protected]!/bootstrap.js:420:24 itemsEndpoint@resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///C:/Users/alexanderkaye/AppData/Roaming/Zotero/Zotero/Profiles/79uiu4p8.default/extensions/[email protected]!/bootstrap.js:419:13 handleErrors/<@resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///C:/Users/alexanderkaye/AppData/Roaming/Zotero/Zotero/Profiles/79uiu4p8.default/extensions/[email protected]!/bootstrap.js:300:18 Zotero.Server.DataListener.prototype._processEndpoint<@chrome://zotero/content/xpcom/server.js:526:26 Zotero.Server.DataListener.prototype._headerFinished@chrome://zotero/content/xpcom/server.js:302:5 Zotero.Server.DataListener.prototype.onDataAvailable@chrome://zotero/content/xpcom/server.js:208:7

(5)(+0000001): HTTP/1.0 500 Internal Server Error X-Zotero-Version: 6.0.27 X-Zotero-Connector-API-Version: 2 Content-Length: 0

(5)(+0002057): GET /zotxt/items?easykey=sagiv_DeepBlueNotes_2015 HTTP/1.1 Host: localhost:23119 Accept-Encoding: gzip

(5)(+0000009): HTTP/1.0 400 Bad Request X-Zotero-Version: 6.0.27 X-Zotero-Connector-API-Version: 2 Content-Type: application/json; charset=UTF-8 "sagiv_DeepBlueNotes_2015 must be of the form DoeTitle2000 or doe:2000title"

(3)(+0016322): {better-bibtex} +29608 idle: Thu Oct 26 2023 23:46:33 GMT-0400 (Eastern Daylight Time), save-database idle

[ganko-pi]

I looked into retorquere/zotero-better-bibtex@v6.7.131...v6.7.132#diff-3bbc877b473fc9c2c1cffd4b254751bcfd238202c5c3beef01ec0006488b2627L133 and saw that Zotero.BetterBibTeX.KeyManager.keys.findOne is now Zotero.BetterBibTeX.KeyManager.get or Zotero.BetterBibTeX.KeyManager.first.

[newhallroad]

Yes, that seems like it might be the root of the problem.
I don't think I have the ability to fix this myself in the code. (Though if someone gave me basic instructions I could try.) It would be great if someone could fix the code so we can use the latest version of BetterBibtex. Thank you.

[retorquere]

See #42

[retorquere]

The PR should fix it, but @egh will have to verify/merge/release.

[newhallroad]

Thanks for your work @retorquere. I tried the update and something is still not working for me. I get this in the debug log:

(5)(+0040669): GET /zotxt/items?betterbibxkey=yerushalmi_Introduction_2013 HTTP/1.1 Host: localhost:23119 Accept-Encoding: gzip

(5)(+0000007): HTTP/1.0 400 Bad Request X-Zotero-Version: 6.0.30 X-Zotero-Connector-API-Version: 2 Content-Type: application/json; charset=UTF-8 "No param supplied!"

zotxt is a wonderful package. I have found it very helpful when using emacs with zotero and pandoc. But unfortunately something is broken. I would be very grateful if @egh was able to look into this and hopefully find a fix.

[retorquere]

I get this in the debug log:

That is because you are still using betterbibxkey in the URL. That has never worked. Change the URL to

/zotxt/items?betterbibtexkey=yerushalmi_Introduction_2013

[newhallroad]

Thanks. Good catch. I corrected the typo and made the pinned citation key standard syntax just for good measure. Still no joy, however. This is the current log:

(5)(+0011687): GET /zotxt/items?betterbibtexkey=yerushalmiIntroduction2013 HTTP/1.1 Host: localhost:23119 Accept-Encoding: gzip

(5)(+0000006): HTTP/1.0 500 Internal Server Error X-Zotero-Version: 6.0.30 X-Zotero-Connector-API-Version: 2 Content-Type: text/plain; charset=UTF-8 Zotero is not definedfindByCitationKey@chrome://zotxt/content/modules/Core.jsm:216:11 itemsEndpoint/<@resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///C:/Users/alexanderkaye/AppData/Roaming/Zotero/Zotero/Profiles/79uiu4p8.default/extensions/[email protected]!/bootstrap.js:418:24 itemsEndpoint@resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///C:/Users/alexanderkaye/AppData/Roaming/Zotero/Zotero/Profiles/79uiu4p8.default/extensions/[email protected]!/bootstrap.js:417:13 handleErrors/<@resource://gre/modules/addons/XPIProvider.jsm -> jar:file:///C:/Users/alexanderkaye/AppData/Roaming/Zotero/Zotero/Profiles/79uiu4p8.default/extensions/[email protected]!/bootstrap.js:300:18 Zotero.Server.DataListener.prototype._processEndpoint<@chrome://zotero/content/xpcom/server.js:526:26 tryCatcher@resource://zotero/loader.jsm -> resource://zotero/bluebird/util.js:16:16 module.exports/PromiseSpawn.prototype._promiseFulfilled@resource://zotero/loader.jsm -> resource://zotero/bluebird/generators.js:97:18 module.exports/Promise.coroutine/<@resource://zotero/loader.jsm -> resource://zotero/bluebird/generators.js:201:9 Zotero.Server.DataListener.prototype._headerFinished@chrome://zotero/content/xpcom/server.js:302:5 Zotero.Server.DataListener.prototype.onDataAvailable@chrome://zotero/content/xpcom/server.js:208:7 

[retorquere]

That doesn't seem to list the actual error that occurred.

[newhallroad]

That was the entire thing. I'm not sure how to interpret it.
Could Zotero is not defined be significant?

[retorquere]

Oh I had missed that. PR updated.

[newhallroad]

Bravo! That did it. pandoc-zotxt.lua is now working as expected, as far as I can see. I hope your code is accepted in an official update.

There is one other glitch. I'm going to post it here in case it is also relevant to changes in better bibtext. If it isn't, please let me know and I'll open another issue.

zotxt should work with zotxt-emacs to autocomplete citekeys. This is not working. As far as I can tell, the mechanism is to trigger a call to the zotxt api complete. (See test file here.) That call is triggered, but it returns an error of Option easykey is required.

The same error occurs from the cli. Generally, the cli seems to be working. For example, curl http://127.0.0.1:23119/zotxt/version returns:

{
  "version": "6.0.2"
}

And curl http://127.0.0.1:23119/zotxt/items?citekey=yerushalmiIntroduction2013 correctly returns the json bibtex info for the appropriate citation. (If I use either of these URIs in the browser, they return Request not allowed, but I assume that's a different issue.)

But curl http://127.0.0.1:23119/zotxt/complete?citekey=@yer returns "Option easykey is required."

Replacing citekey with easykey - curl http://127.0.0.1:23119/zotxt/complete?easykey=@yer - returns no error, but an empty line; it doesn't find the entry. In any case, I can't change the way that emacs autocomplete makes the call.

The zotero log for the failed autocomplete is below. As I said, please let me know if this is not in your realm, or belongs in another conversation, and I'd be happy to open another issue.

Most of all, thank you for your responsiveness and generosity to the community.

(5)(+0006227): GET /zotxt/complete?citekey=yer HTTP/1.1 Host: 127.0.0.1:23119 User-Agent: curl/8.4.0 Accept: */*

(5)(+0000002): HTTP/1.0 400 Bad Request X-Zotero-Version: 6.0.30 X-Zotero-Connector-API-Version: 2 Content-Type: application/json; charset=UTF-8 "Option easykey is required."

(5)(+0000146): GET /zotxt/version HTTP/1.1 Host: 127.0.0.1:23119 User-Agent: curl/8.4.0 Accept: */*

(5)(+0000001): HTTP/1.0 200 OK X-Zotero-Version: 6.0.30 X-Zotero-Connector-API-Version: 2 Content-Type: application/json; charset=UTF-8 { "version": "6.0.2" }

[retorquere]

That call is triggered, but it returns an error of Option easykey is required.

easykey is a required parameter in that URL call. As far as I can tell this has always been the case.

If I use either of these URIs in the browser, they return Request not allowed, but I assume that's a different issue.)

Zotero does not allow browsers to call the built-in server.

Replacing citekey with easykey - curl http://127.0.0.1:23119/zotxt/complete?easykey=@yer

That endpoint doesn't understand citation keys.

this is not in your realm

It's sort of not. I am happy to assist in getting this sorted out, but this issue has sat unaddressed for close to 3 months; it doesn't seem like @egh is still involved, and I don't even know what this extension does. And AFAICT, noone but @egh can even merge these changes.

[retorquere]

Most of all, thank you for your responsiveness and generosity to the community.

Ah thank you, and you're very welcome. I find it very rewarding work.

[retorquere]

@egh still seems pretty active on gitlab, and his github profile shows his public contact information. Maybe you could reach out to him that way? He may just not be monitoring github.

[retorquere]

If some of zotxt's functionality aligns closely with BBT, I am open to taking them into BBT, but I don't want to nick zotxt's ideas without consideration of what @egh wants.

[retorquere]

Has anyone tested whether this works on zotero 7 beta? The bootstrap code doesn't look at all like what zotero itself proposes to use. If it doesn't run o 7, its going to be obsolete soon.

[newhallroad]

Well I for one would be delighted to see the continuation of this project, either as it stands or as part of BBT. I have basic coding skills, but not enough to develop the project. I'd happily be a guinea pig for testing, though.
I'll try to reach him via his info on github and hope he is able to respond.

[egh]

Apologies for the radio silence! Thanks to @retorquere I think we have a fix that I've just released.

As for Zotero 7, I have a branch for that, which is being worked on.