Is Your Knowledge Base Ready for AI? A 12-Point Checklist

Score your knowledge before building the AI layer.

Most AI projects that fail do not fail on the model. They fail because the knowledge underneath was inaccurate, stale, unowned, unstructured, unreadable-by-permission or untraceable — and the AI layer faithfully amplified all of it. Readiness assessments have become a product category of their own (PwC runs enterprise AI readiness assessments with scoring and roadmaps), but you can get most of the diagnostic value yourself, in one honest hour.

Below is a 12-point checklist across six dimensions. This is a practical KifferLiu assessment framework, not an industry certification — its value comes from being brutally honest when you score it.

Why AI readiness starts with knowledge

An AI system does not create knowledge. It retrieves, recombines and phrases what already exists. That property is symmetrical:

Governed knowledge  →  AI that sounds like your best engineer
Ungoverned knowledge →  AI that sounds like your worst intern,
                        with total confidence

The system inherits the state of your knowledge. Every dimension below is therefore scored before choosing a model, a vector database or an agent framework — those choices get easier, cheaper and more effective once the foundation is honest.

The 12-point AI knowledge readiness checklist

Score each checkpoint: 0 (no), 1 (partly), 2 (yes, verifiably). Maximum 24.

1. Accuracy

• We can vouch that the core facts in our knowledge base (specs, prices, policies) are correct
• Known conflicts between documents are identified and resolved, not just coexisting

2. Freshness

• For any given document or fact, we can tell which version is current
• There is a working mechanism — an owner or a process — that updates knowledge when reality changes

3. Ownership

• Every knowledge domain (products, pricing, support, policies) has a named owner
• When something is wrong, anyone in the company knows who fixes it

4. Structure

• Individual facts, products and policies can be retrieved — not just whole documents
• Terminology is consistent across departments (sales, engineering, ERP do not use three names for one product)

5. Permissions

• We can determine, per document or data set, who is allowed to see it
• Cost data, personal data and public specs are separated rather than mixed in one store

6. Provenance

• Answers can be traced to a dated source document
• Customers or auditors could verify a claim against that source

Scoring

 0–8    Not AI-ready       fix the foundation before any deployment
 9–16   Partially ready    deployable on a governed slice only
17–20   Good foundation    build, with governance as ongoing work
21–24   AI-ready           the AI layer is now the easy part

Two dimensions deserve extra weight in practice: freshness and permissions. Stale knowledge silently corrupts every downstream answer, and missing permissions are the one problem that can turn an internal tool into an incident.

How to run the assessment

The number is only as good as the session that produced it:

• Right people in the room: whoever owns the knowledge (often not IT), whoever owns the systems, and one skeptic. The skeptic’s job is to ask “prove it” after every 2.
• Evidence, not opinions. A “2” for freshness means someone can show, right now, which version of a given spec is current. If the room debates it, it is a 1.
• One domain at a time. Scores across “all our knowledge” average away exactly the differences you need. Score the domain you intend to deploy on first.
• Re-score quarterly. Readiness is a stock that depletes; the second measurement is where the trend appears.

Run honestly, the exercise usually takes an hour and produces something a twelve-week consulting engagement would render in a deck: a short, ranked list of what is actually broken.

What a bad knowledge base looks like

The pre-assessment state we usually find at mid-size manufacturers:

• The same product spec exists as a 2019 PDF, a 2022 spreadsheet and a website page — all three still linked
• Half the indexed documents describe products no longer shipped
• Cost data lives in the same folders as public specifications
• “XR line”, “G2 series” and the ERP code are the same product, per three departments
• Nobody can name the owner of any of it

If four or more of those sound familiar, you are in the 0–8 band — and that is useful information. It is dramatically cheaper to learn it now than in a post-mortem.

What AI-ready knowledge looks like

The same manufacturer, after governance:

• One current, owned, dated version per document; superseded versions archived, not indexed
• Facts retrievable individually, with consistent terminology mapped across departments
• Access levels attached to content, not to folders nobody checks
• Every answer carries a citation to a verifiable source
• A named person reviews the knowledge slice the AI depends on, on a fixed cadence

None of this requires new technology. It requires decisions — and someone accountable for them.

Why RAG doesn’t fix bad knowledge

One sentence, worth pinning somewhere visible:

RAG can retrieve bad information faster. It cannot make bad information true.

Retrieval amplifies whatever is most retrievable. If the 2019 PDF phrased things more cleanly than the 2024 correction, guess what the system serves. Every failure mode in the “bad knowledge base” list above survives a RAG deployment intact — it just reaches customers at machine speed and with fluent confidence.

This is why governance precedes architecture: the disciplines overlap heavily with knowledge governance, and the whole stack — retrieval, permissions, memory, tools — is decided upstream in context engineering.

What to fix first

If your score is low, do not start everywhere. In order of return:

1. Pick one workflow (support, quoting, spec lookup) — never govern “all knowledge” at once
2. Deduplicate and version that slice; archive what is dead
3. Name an owner for it — this single decision outlives every tool
4. Separate what must not leak (cost, personal data) before anything gets indexed
5. Add citations from day one, so every answer keeps proving itself

Then, and only then, choose the retrieval architecture — including whether you need agentic RAG at all.

Score your knowledge before building the AI layer.

A governed slice of knowledge beats an ambitious pile of documents, every time. We run this assessment as part of every deployment: see how we build business AI systems.