I had just added six commentary volumes to my Bible search. I asked it a question about the fruit in the garden of Eden. The answer came back with sermons and Bible verses. Not one of the six new books was among the sources.
The books were in the database. The search just did not pick them.
The Need: A Source You Added Must Be Able to Appear
If a new source never shows up, you cannot tell whether it is broken or simply ranked low. You only see the answer, and the answer looks fine. It is easy to think “the system works” and move on.
My question was: what is the essence of the fruit event in Genesis? (In Korean, one short line.) Genesis 3 is the center of the commentary on that book. If any new book should appear, it is this one.
The Proof: Same Question, Before and After
Before the fix, the source list had sermons and verses only.
After the fix, I ran the same question again with the model switched off, so I could see only what the search returns:
[1] sermon 00:19:39 (concept: the fruit; relation)
[2] sermon 00:21:48
[3] sermon 00:23:54
[4] commentary on Genesis, chapter 3, paragraph 75 (vector)
[5] sermon ...
[9] commentary on Leviticus, chapters 4-5 (vector)
[10] commentary on Exodus, chapters 5-6 (vector)
[11] commentary on Leviticus, chapters 5-6 (vector)
Four of eleven sources are now commentary. The Genesis 3 paragraph is at number 4. I did not read each commentary result closely, so I cannot say the Leviticus and Exodus ones are all useful. They are at least about sin offerings and sin, which is near the topic.
I only tested a handful of questions. I do not have a set of questions with known good answers, so I cannot give you a recall number.
The Story: Why the Books Were Missing
The library has two big groups:
| Type | Chunks (about 700 characters each) |
|---|---|
| Sermons (two speakers) | 7,114 |
| Commentary (six translated volumes) | 1,990 |
The search finds the 8 chunks nearest to the question by meaning. Sermons are almost four times more numerous, and they talk about the same topics in plain speech. So the top 8 were all sermons. The commentary did not lose by a small margin. It was not in the list at all.
I found a second problem while looking. The question contains the word “Genesis”. The search model turned that into a pull toward the introduction of the Genesis book, and other chunks that merely mention the book name. That is not what I meant.
The How: Three Small Changes
All three are in query.py. The code is short, so I will show the idea.
1. Pick the top few per source type, not the top few overall.
types = np.array([r["source_type"] for r in rows])
per_type = max(3, k_vec // 2) # 4 when k_vec is 8
for t in np.unique(types):
idx = np.where(types == t)[0]
for i in idx[np.argsort(-sims[idx])[:per_type]]:
add(rows[i], "vector", float(sims[i]))
Each type gets its own four slots. A type with fewer chunks can no longer be pushed out by a bigger one. The answer step already limited each type to 7 slots out of 14, so this matches it.
2. Remove book names from the text I embed.
qe = q
for b in sorted(refs.BOOKS, key=len, reverse=True):
qe = qe.replace(b, " ")
The book name is not useful for meaning. If what is left is shorter than 4 characters, I use the original question.
3. Use the book name as a small bonus instead.
If the question names a book, chunks from that book that are already in the result list get +0.2. It only reorders what was found. It never adds new results.
🗂 Claude.md Rule When one source type has many more chunks than another, a single top-k vector search will be filled by the big one. Pick the top few per type, then merge.
💬 Prompt that worked (2026-10-23 07:28) “When I asked ‘what is the essence of the fruit event in Genesis’, the commentary books do not appear as sources?”
I did not say “fix the ranking”. I described what I expected to see and did not see. That made it a recall problem and not a wording problem.
Two smaller fixes the same day
- Source cards showed the opening text twice. The collapsed card showed the first 260 characters. When I expanded it, it showed all the text, including those same 260 characters, so the beginning appeared two times. Now the collapsed part ends with “…” and the expanded part holds only the rest. I noticed it only because I clicked “expand” on a real answer. A test with short text would not have shown it.
- Clear errors for a broken tunnel. If the GPU box cannot be reached, the page now says so and hints to check the tunnel. A missing model gives a different message. Before, both looked like a bare 404.
What I decided not to build
I had asked Claude to run concept and relation extraction over the whole library. It reads three chunks at a time, lists concepts, and links them with relations such as foreshadows and fulfills. For 9,104 chunks that is about 3,200 groups. At about 50 seconds each on the 32B model, I estimated roughly 44 hours.
Then I asked a different question.
💬 Prompt that worked (2026-10-23 06:46) “Why do we need to extract concepts and relations? Even now, when I ask ‘what is the relationship between the bronze serpent and Christ’, the commentary on Numbers is found along with the other sources.”
That was the right question. Plain vector search and verse links already answered it. Claude agreed, and I did not run the full extraction. I ran it on 26 chunks only (about 46 concepts and 21 relations) as a test. Those few concepts already show up in the results above as the label next to the first three sermons. I will revisit it only if I find questions that vector search cannot answer.
Cost
- No new API cost this time. The search runs on my PC and on a borrowed GPU box.
- The six translated volumes cost about $1.20 earlier, in a separate project.
- Claude Code is on my existing subscription.
Still open
- The word “fruit of good and evil” (as the sermons say it) does not appear in the translated Genesis book. The translation says “the tree of knowledge”. My search found chapter 3 by meaning, but a word search would miss it. A glossary and re-translation would fix it. I have not done that.
- Bible quotes inside the translated books are the model’s paraphrase, not the Korean Bible text I use elsewhere.
- Korean spoken numbers for verses (“chapter ten, verse twenty-six”) are still not recognized.
- Auto-start after a reboot is set up but I have not seen it work yet.
- No test set. Right now “it works” means “I asked a few questions and looked”.