Repository navigation
Conversation
Add scholarly/mcp.py and a `scholarly-mcp` console script that run scholarly as a Model Context Protocol server, installable with `pip install scholarly[mcp]`. - Seven read-only tools wrapping the public API: search_authors, get_author, search_publications, get_publication, get_bibtex, get_citing_publications and get_related_articles. They take plain names, ids and titles and return plain JSON, as MCP clients cannot hold Author/Publication objects. - stdio, sse and streamable-http transports; --proxy none|free|tor|scraperapi reusing ProxyGenerator. Secrets come from the environment, never flags. - Supports both the 1.x (FastMCP) and 2.x (MCPServer) MCP Python SDKs. - Calls run one at a time on a single worker thread, since scholarly is a stateful singleton and Google Scholar penalises bursts. - The MCP SDK is an optional extra (Python 3.10+); core scholarly is unchanged. - Add offline unit tests, docs and a CI step (SDK tests skip on Python 3.8). Fixes scholarly-python-package#592
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3edf01a551
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ok = pg.Tor_External(socks_port, control_port, env.get("SCHOLARLY_TOR_PASSWORD", "")) | ||
| if not ok: |
There was a problem hiding this comment.
When --proxy tor is pointed at an unavailable SOCKS service, ProxyGenerator.Tor_External() returns a non-empty status dictionary with proxy_works=False, rather than False. This condition therefore accepts the failed setup and installs a generator whose session is still direct; publication requests can then leave via the user's IP even though the CLI claims it will refuse to start unproxied. Test the returned proxy_works field (and raise ProxyConfigError) before calling api.use_proxy.
Useful? React with 👍 / 👎.
| def get_citing_publications(self, title: str, limit: Optional[int] = None) -> Dict[str, Any]: | ||
| """List publications that cite the best-matching publication for a title.""" | ||
| pub = self._find_publication(title) | ||
| return self._listing(title, _take(self.api.citedby(pub), _clamp_limit(limit, 10))) |
There was a problem hiding this comment.
When the best match has zero citations, PublicationParser._scholar_pub() records num_citations=0 but does not add citedby_url; _Scholarly.citedby() then reaches PublicationParser.citedby(), which indexes that missing key. Thus a common valid get_citing_publications call is reported as an internal tool error instead of a zero-count listing. Short-circuit on the citation metadata before invoking citedby.
Useful? React with 👍 / 👎.
Stopping a network transport with Ctrl+C printed a CancelledError traceback and exited with status 1. Treat it as the normal shutdown it is and exit 0.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cd31563abb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| pg = ProxyGenerator() | ||
| if mode == "free": | ||
| ok = pg.FreeProxies() |
There was a problem hiding this comment.
When --proxy free runs on a network where no candidate proxy works, ProxyGenerator.FreeProxies() raises MaxTriesExceededException (it does not return False), so control never reaches the if not ok conversion below. Since main() catches only ProxyConfigError, the documented startup check exits with an uncaught traceback/status 1 instead of the concise configuration failure/status 2; catch and wrap the setup exception here.
Useful? React with 👍 / 👎.
| transports have **no authentication**, so only bind a non-loopback address on a trusted | ||
| network; the server logs a warning when you do. | ||
| ``--proxy`` | ||
| ``none`` (default), ``free``, ``tor`` or ``scraperapi``. This configures the same |
There was a problem hiding this comment.
In a clean environment following the documented pip install scholarly[mcp], selecting this advertised --proxy tor mode cannot work: the mcp extra in setup.py installs only the MCP SDK, while ProxyGenerator.Tor_External() raises RuntimeError unless the separate stem/tor extra is installed. Document scholarly[mcp,tor] for this mode (and preferably translate missing support to ProxyConfigError) so the listed option is usable.
Useful? React with 👍 / 👎.
| def get_related_articles(self, title: str, limit: Optional[int] = None) -> Dict[str, Any]: | ||
| """List articles Google Scholar considers related to the best-matching publication.""" | ||
| pub = self._find_publication(title) | ||
| return self._listing(title, _take(self.api.get_related_articles(pub), _clamp_limit(limit, 10))) |
There was a problem hiding this comment.
When the best match is a valid result whose Scholar snippet omits the optional “Related articles” link, PublicationParser._scholar_pub() leaves out url_related_articles, and _Scholarly.get_related_articles() indexes that key. This tool therefore reports an internal error instead of the empty listing promised for no results; check the publication metadata before calling the API and return _listing(title, []) when the link is absent.
Useful? React with 👍 / 👎.
|
|
||
| To expose `scholarly` to LLM agents and other [Model Context Protocol](https://modelcontextprotocol.io/) clients, install with the `mcp` tag as | ||
| ```bash | ||
| pip3 install scholarly[mcp] |
There was a problem hiding this comment.
On macOS's default zsh, the unquoted scholarly[mcp] is parsed as a filename-generation pattern and, with the usual no-match case, the command aborts before pip runs. The Tor instructions immediately above already quote their extra for this reason; quote this command and the other new README/docs install examples so MCP installation works when pasted into zsh.
Useful? React with 👍 / 👎.
Fixes #592
Description
Adds an optional MCP (Model Context Protocol) server, so a pip-installed
scholarlycan be launched as a tool server for LLM agents and MCP clients:What is added
scholarly/mcp.pyplus ascholarly-mcpconsole script and anmcpextra insetup.py.search_authors,get_author,search_publications,get_publication,get_bibtex,get_citing_publications,get_related_articles. They take plain names, ids and titles and return plain JSON, because an MCP client cannot holdAuthor/Publicationobjects between calls.--transport {stdio,sse,streamable-http},--host,--port,--proxy,--log-level.docs/mcp.rst, README section), unit tests (test_mcp.py) and a CI step.Design decisions worth a look
import scholarlyis unaffected andscholarly-mcpexits with code 2 and an install hint.FastMCPtoMCPServerand movedhost/portfrom the constructor torun(), so a plainpip install scholarly[mcp]would otherwise break on one of them.scholarlyis a stateful singleton (shared session and proxy) and Google Scholar penalises bursts, so concurrency would only make blocking more likely. This is covered by a test.SCRAPER_API_KEY,SCHOLARLY_TOR_*), never from flags, so they do not appear in process listings. If the requested proxy cannot be set up the server refuses to start rather than silently running unproxied.MaxTriesExceededException,DOSException) are raised as the SDK'sToolErrorso the agent sees the message. This matters on SDK 2.x, which deliberately withholds the text of any other exception from the client. Unexpected exceptions are still reported as errors.note. Google Scholar sometimes answers a blocked client with a sign-in or CAPTCHA page, whichscholarlyparses as "no results". Without the note an agent would read that as "no such author". I hit exactly this while testing: callingscholarly.search_author("Marie Curie")directly (no MCP involved) returned[]on my network, and the logs show Google redirecting the request toaccounts.google.com/Login.127.0.0.1by default and log a warning for any other address, since there is no authentication.--bind/--listen; this uses the conventional--host/--port.Testing
python -m unittest -v test_mcp) using a fake API, so nothing is sent to Google Scholar. They cover the tool logic and validation, proxy configuration, the CLI, SDK-version dispatch, error surfacing, and serialised execution, and drive a real server object throughlist_tools/call_tool.mcp1.30.0 and withmcp2.3.0. With no SDK installed all 41 run, 8 SDK-dependent ones skip, and the CLI reports the missing SDK.flake8is clean under the repo config.scholarly-mcpscript over stdio on both SDK generations. On 2.x it initialised, listed all 7 tools and made tool calls. On 1.x it initialised and made tool calls, and a Google Scholar failure arrived at the client as a readable tool error (tool listing on 1.x is covered by the in-process tests).docsbuild with Sphinx; the only warning is an existing one indata_types.py.Not verified, so reviewers know
ast.parse(feature_version=(3, 8))), not executed. By design the 8 SDK tests skip there.bibtexparser, see Incompatible with bibtexparser 2.0.0b7 #546, which I left alone). An SDK job on Python 3.10+ would be a sensible follow-up.Checklist
developand notmain.The new tests pass (see Testing). I did not run
test_module.py, which needs network access andproxies; this PR does not modify any existing library code.