building a legal chatbot for a Bulgarian law firm without overengineering it

How we replaced a five-service RAG architecture with FastAPI, LangChain, and pgvector — and why it was the right call for this client.

The Client Problem

A Bulgarian law firm needed a chatbot. Not a generic one. One that answers specific questions about company registration law — minimum capital requirements, name-change procedures, shareholder liability — in plain language, with citations, in Bulgarian.

Their users are not lawyers. They are business owners trying to understand whether they need an accountant or a solicitor. They ask questions like "How much money do I need to start an LLC?" and expect a straight answer, not a 12-page legal memo.

The firm had two constraints that shaped everything: a small internal IT team and no appetite for managing infrastructure they did not understand.

We could have built the "right" production RAG system. Pinecone for vector storage. Redis for caching. Celery workers for async embedding pipelines. LangGraph for state management. Kubernetes somewhere behind all of it.

We did not. Here is what we built instead — and why.

The Architecture Decision

Before writing a single line of code, we asked a question we ask every client: What does this system need to do in the next 12 months, not the next 5 years?

The answers were concrete:

That last point settled the vector database debate immediately. The client runs PostgreSQL. They know how to back it up, monitor it, and restore it. Introducing a managed vector service meant introducing a new vendor, a new billing account, a new failure domain, and a new thing for their team to learn.

pgvector is a PostgreSQL extension. It lives in the same database, uses the same connection pool, and appears in the same backup. For a corpus of a few thousand legal articles, it performs well enough.

User Question (Bulgarian)
         │
FastAPI /chat endpoint
         │
   ┌─────▼─────────────────────────────────┐
   │  1. Intent gate (GPT-4 system prompt) │
   │  2. Query embedding (OpenAI)          │
   │  3. Vector search (pgvector / SQL)    │
   │  4. Context assembly (top 5 chunks)   │
   │  5. Answer generation (GPT-4 Turbo)  │
   └─────────────────────────────────────┘
         │
Bulgarian Response + Source Citations

Five steps. One database. One service to deploy.

Document Management: One Endpoint to Rule Them All

The firm's admins are not engineers. They needed to upload a legal document and have it immediately searchable — without understanding what an embedding is.

We built a single endpoint: POST /documents/upload-with-chunks.

The admin sends a JSON body:

{
  "title": "Търговски закон - ЕООД",
  "content": "Чл. 113. Еднолично дружество с ограничена отговорност...",
  "law_reference": "Търговски закон",
  "source_url": "https://lex.bg/..."
}

Behind that endpoint, in a single database transaction:

If embedding generation fails, nothing persists. No partial states. No "document uploaded but not searchable" support tickets on Monday morning.

The chunking strategy is domain-specific by design. Bulgarian legal documents follow a predictable structure — articles are numbered sequentially and clearly delimited. Splitting on Чл. XXX produces chunks that map to real legal concepts, not arbitrary token windows. The LLM retrieves and cites whole articles, not fragments of them.

The RAG Chain: Kept Intentionally Thin

LangChain handles the pipeline. We kept the chain to five sequential steps with no agents, no tools, no branching logic.

Step 1 — Intent gate. The system prompt instructs GPT-4 to refuse anything outside Bulgarian company registration law. Users asking about tax returns or property law receive a polite redirect. No separate classifier needed.

Step 2 — Query embedding. The user's question is embedded with text-embedding-3-large. The same model used for document chunks — no embedding space mismatch.

Step 3 — Vector search. A single SQL query with cosine distance:

SELECT
    chunk_text,
    article_reference,
    1 - (embedding <=> $1::vector) AS similarity
FROM document_chunks
WHERE embedding <=> $1::vector < 0.3
ORDER BY similarity DESC
LIMIT 5;

Threshold of 0.3 filters noise. Top 5 chunks give the LLM enough context without exhausting the context window.

Step 4 — Context assembly. Retrieved chunks are formatted with article references so the LLM can cite them precisely:

Чл. 117. [чл_117]: Минималният капитал е 2 лв.
Чл. 10.  [чл_10]:  Фирмата може да се променя...

Step 5 — Answer generation. GPT-4 Turbo generates a response at temperature 0.3. The system prompt instructs it to use plain language, cite sources in the format "Според чл. 117 от Търговския закон...", and never pad answers with irrelevant information.

The entire chain is roughly 200 lines of Python. Any engineer on the team can read and modify it.

Where We Made Deliberate Compromises

We are transparent with clients about what they are not getting.

No vector index for high-dimensional embeddings

text-embedding-3-large produces 3,072-dimensional vectors. pgvector's HNSW index caps at 2,000 dimensions. We are doing sequential scans.

At 1,000 documents, searches complete in under 100 milliseconds. We set a monitoring threshold at 200ms. If we consistently breach it — which would indicate a corpus growing toward 10,000+ documents — the options are to switch to text-embedding-3-small (1,536 dimensions, indexable), implement IVF approximate search, or migrate the vector layer to a dedicated service. The interface in application code does not change: search_similar_chunks() takes a query, returns context.

Rate limits during bulk uploads

Uploading 100 legal articles triggers OpenAI's rate limits. We implemented exponential backoff with jitter — initial delay of 1 second, multiplied on each retry, with up to 1 second of random jitter. Bulk upload takes 10–15 minutes for 100 articles. For a firm that updates its document corpus quarterly, this is acceptable. If the cadence increases, a proper job queue is the next step.

Conversation context has a ceiling

We truncate conversation history beyond the last 5 user-assistant pairs. Users lose context from earlier in a long session. In practice, legal Q&A sessions are short and focused — users ask 3–5 related questions, get their answers, and leave. The truncation has not surfaced as a complaint. If it does, summarization or session-scoped vector memory are the natural next steps.

No citation verification

The LLM cites article numbers in responses. We do not verify that citations are accurate. A hallucinated citation is indistinguishable from a real one in the UI. A post-generation verification step — looking up cited articles in the database and comparing — is on the roadmap.

No streaming

Responses return in full after 2–5 seconds. No progressive rendering. FastAPI supports StreamingResponse and we can add it, but the client's users did not find the latency disruptive in testing. Legal answers are read carefully, not skimmed in real time.

Security and Operations

Document upload endpoints are protected with HTTP Basic Auth. Two people at the firm need access. There is no public registration flow, no third-party integrations, no token refresh logic to manage. Basic Auth is exactly right for this use case.

Rate limiting uses in-memory session tracking. No Redis. For 100 queries per day, Redis would be an operational cost with no measurable benefit.

Deployment is a single docker-compose up --build. One container. One log file. One health check. When something breaks — and it will — the firm's IT contact opens one log, finds the error, and either fixes it or calls us.

Error responses are in Bulgarian. Users see "Не мога да намеря релевантна информация за този въпрос" rather than a stack trace.

The Economics

The traditional production RAG stack — managed vector service, Redis, Celery, separate embedding microservice, monitoring for each — runs at a minimum of €100–500 per month before accounting for the engineer who operates it.

This system costs what the client already pays for PostgreSQL, plus OpenAI API usage proportional to actual queries. At 100 queries per day with average context assembly, OpenAI costs are in the range of €20–50 per month.

More importantly: there is no new service to monitor, no new vendor to negotiate with, and no new technology for the client's team to learn. When we hand this off, they are operating a FastAPI app backed by a database they have run for years.

When This Architecture Fits

It works well when:

It doesn't fit when:

The signal to re-evaluate: vector searches consistently exceeding 200ms, corpus hitting 50,000+ documents, or daily bulk upload operations becoming the norm. When those signals appear, the migration path is straightforward — the application code already abstracts the vector layer behind a single function call.

What the Client Got

A Bulgarian legal chatbot that:

It went from concept to production in six weeks. The client's admin team uploaded 200 legal articles on day one without assistance. The first real user query was answered correctly, with the right citation, in 3.1 seconds.

The system described here follows codebefore's core principle: prove the simplest thing that works before adding complexity. When traffic grows and the corpus expands, the architecture evolves. For now, PostgreSQL is doing the job — and nobody is paying for infrastructure they do not need.

Sometimes the best architecture is the one you don't have to operate.