The deadline is part of the answer
We spent three months building a conference recommender that worked. Its suggestions were relevant and most of them were useless, because recommending a venue is a freshness problem before it is a matching problem.
LLMs · Retrieval · Recommender systems
Our supervisor kept running into the same thing. A researcher would finish a paper, or most of one, and then have no idea where to send it.
That sounds like the easy half of the job. It is not. You need to know which venues cover your topic, which ones take work at your level, what the tracks are, whether submission is still open, when it closes, and whether the page you are reading describes this year or three years ago. The usual method is to open a dozen tabs and read call-for-papers pages until something fits — and to hope you guessed the right words to search for in the first place, because if the venue describes your subject differently than you do, you will never see it.
So we built something for it. This was a team project — five of us and a supervisor — and it grew into a larger platform we ended up calling ScholarNexus. My part was the conference recommendation, and that is what this is about.
Three months on the wrong shape of problem
The first version was the obvious one.
Collect a dataset of conferences and their topics. Fine-tune a model on it. The user uploads a paper, the system works out what it is about, and matches that against what it learned. A classification problem with a nice clean shape, the kind you can describe on a slide.
Concretely: just under eighteen thousand abstract-and-venue pairs, scraped from public repositories and reaching back to 1990, used to fine-tune a LLaMA 3.2 1B with LoRA adapters so that it would answer an abstract with a conference name. Once it had named one, a second stage looked that conference up on WikiCFP and summarised the call for papers — dates, location, deadline, official link.
We worked on that for about three months. That system is the one our published paper describes, and the paper is where its details belong: AI-Powered Conference Recommendation: Fine-Tuning Language Models for Context-Aware Academic Support, SRC 2025, 10.1109/SRC65875.2025.11263514. This article is about what came after it.
And it worked. You could hand it an abstract and get back conferences that were genuinely about the same subject. As a matching problem it was solved.
It was right, and it was useless
The conferences it suggested had already happened.
Not all of them, but enough. The model knew a venue was a good fit for the topic. It had no way of knowing the deadline had passed four months ago. Every fact it had about the world was frozen at the moment we collected it, and conferences are not a stable subject — the entire useful content of a call for papers is a date in the near future.
The retrieval stage did not save us, and it is worth being exact about why. The system did go out to the web — but only after the model had already chosen which conference to name, and it chose from what it had memorised. Looking up a decision that has already been made can describe that decision more accurately. It cannot make it a different decision.
A venue that suits your paper perfectly and closed last spring is not a worse recommendation than the alternatives. It is not a recommendation at all. The user cannot act on it. Correct and unusable turn out to be the same thing here.
That took a while to accept, because the failure did not look like a failure. Nothing crashed. The relevance was fine. The output looked exactly like success, which is the most expensive kind of bug.
The deadline is part of the answer
Once that landed, the problem changed shape.
We had been treating conference recommendation as matching: here is a paper, here is a set of venues, find the closest. But matching is only half of it. The other half is retrieval, and it is time-sensitive. A recommender whose knowledge is fixed at training time cannot be correct about a moving target, no matter how good the matching is.
So the deadline stopped being metadata attached to the answer. It was part of the answer. A venue with no open call is not a candidate.
That is a different system, and it meant giving up on the part we had spent three months building.
What we built instead
The final version does not learn conferences at all. It goes and looks.
You give it a paper as a PDF, or just a title and abstract. The text comes out with PyPDF2, gets encoded with Hugging Face embedding models, chunked, and indexed in a FAISS vector store — so what the system holds is a semantic representation of your work rather than a bag of your keywords.
Then a search agent goes out to the live web through the Tavily API and collects what is actually being announced right now: conference names, tracks, locations, submission deadlines, official URLs. Those get encoded the same way, compared against the paper by cosine similarity, and ranked.
After that you can talk to the results. Ask when the deadline is, or whether your paper suits a particular track, and the answer comes from what was retrieved rather than from what a model remembers. That part runs on Mistral-Nemo-Instruct with LangChain handling conversation memory, so you do not have to restate which conference you mean in every question.
The knowledge moved out of the weights and into the retrieval step. That is the whole change, and it is the only reason the system stopped giving confident answers about events that were already over.
What "context-aware" actually means here
The phrase is in the title of our paper and it is doing three separate jobs.
The first is the paper. The system reads what you actually wrote rather than a search box, which matters because venues and authors rarely use the same vocabulary for the same subject. Semantic encoding is what lets a venue surface when its call for papers phrases your topic differently than you do.
The second is the conference. Everything the system says about a venue — dates, tracks, location — comes from what was retrieved about that venue, not from the model's memory.
The third is the conversation. Once you are looking at a result, follow-up questions carry the thing you are looking at.
None of those is impressive on its own. Together they are the difference between a search result and something you can ask a question.
What the evaluation does and does not show
We evaluated the conference module by giving it a research abstract, taking the fifteen conferences it returned, and having an LLM score each one from one to five for relevance to the topic. Most scored four or five.
That is a real result and it is a small one. It says the ranking is not producing nonsense. It does not say the system is better than the alternative, because we did not run that comparison on those cases — no head-to-head against keyword search, against a static conference database, or against what a researcher would find on their own in an afternoon.
It also does not say the recommendations were useful. Relevance was scored by a model against a topic. Whether a researcher looked at the list and found somewhere they could actually submit is a different question, and answering it needs researchers, not a scoring prompt.
So: it works well enough on the cases we tried, judged the way we judged them. Anything stronger than that sentence is a claim we did not earn.
And we were not first. Conference recommenders existed, and so did search engines, Semantic Scholar and reference managers. What we were trying to close was the gap between them — full-paper context, live deadlines, and the ability to ask a follow-up, in one place.
Where it still fails
The clearest limitation is the one it inherits from retrieval: if the information does not exist publicly, the system cannot produce it. Ask for venues in a year far enough out that nobody has announced anything and there is nothing to find. That is the correct behaviour and it still reads as a failure to a user.
Grounding reduces hallucination and does not remove it. A vague question against thin retrieval will still get a confident answer.
The evaluation was on computer science and AI papers. I would not assume the same behaviour in medicine or the humanities, where the venues, the vocabulary and the conventions are all different.
And running it was heavy. Several language and vision models, vector indices, live search and video generation in one system meant real memory pressure. On modest hardware that shows up as slowness and timeouts, which to a user is indistinguishable from being broken.
The last week was the hardest
Every module worked. That was the problem.
Each of us had been building a piece — conference recommendation, related work search, integrity analysis, paper analysis, peer-review simulation, the assistant — and each piece was fine in isolation. Putting them into one application meant reconciling APIs, input and output formats, file uploads, error handling, timeouts, and which models were loaded when.
Alongside AI-generated text detection, the project included a humanization feature intended to make flagged passages sound more natural and less machine-generated.
Meanwhile we kept adding things. Every new feature looked small from the outside and none of them were, because each one arrived with integration work, new failure modes and more load on hardware that was already strained.
The last week is not the time to widen scope, and a good idea arriving late is still a bad idea. I had to learn that by doing the opposite.
What I would do today
Not the same way, but not for the reason people usually mean.
Current models are much better at reading papers, holding long context, calling tools and returning structured output. A lot of what we hand-built would now be a few well-specified tool calls, and the prompt engineering around the edges would mostly disappear.
What would not change is the part we got wrong the first time. I still would not let the model answer from memory. Live search, official sources, an explicit check that the deadline has not passed, a citation for every claim, and a refusal when nothing reliable comes back.
If anything I would separate the stages more sharply than we did — retrieval, ranking, generation and validation as distinct steps, so the model is never the thing inventing a venue or a date. It should be reasoning over information that something else went and verified.
That is the lesson that survived. The hard part of a recommender is not the matching. It is knowing what is still true.