Everybody claim that they are experts in AI. Of course they do. But how to measure it?
Level 0: AI Unaware
Has little or no understanding of modern AI tools. May have heard of ChatGPT but doesn’t use AI in their work.
Evaluation: Ask how they currently perform common work tasks such as research, writing, problem-solving, or coding. Person at this level typically do not mention AI tools or are unable to explain how AI could improve their workflow.
Level 1: AI User
Uses AI for simple tasks such as asking questions, writing emails, summarising text, or generating ideas. Mostly copies and pastes prompts.
Evidence: Occasionally uses AI assistants for simple tasks.
Evaluation: Ask to complete a simple task using an AI assistant, such as drafting an email, summarising a document, or generating ideas. Observe whether they can write an effective prompt and critically review the AI’s response instead of accepting it without question.
Level 2: AI Practitioner
Regularly integrates AI into daily work. Understands prompt engineering, evaluates AI output, chains prompts together, and knows each tool’s strengths and limitations.
Evidence: Reliably improves work quality and productivity with AI.
Evaluation: Present a realistic work scenario and ask how they would use AI to complete it. Look for structured prompting, iterative refinement, validation of AI-generated content, and an understanding of when AI should or should not be used.
Level 3: AI Builder
Designs AI-powered workflows or applications. Uses APIs, automation platforms, RAG, agents, embeddings, MCPs, or AI SDKs to solve business problems. Doesn’t necessarily build foundation models.
Evidence: Designs AI-powered workflows and automations.
Evaluation: Ask to explain or design an AI-powered solution for a business problem. They should be able to describe the overall architecture, data flow, model selection, integration points, and trade-offs rather than simply naming AI technologies.
Level 4: AI Engineer
Builds and deploys production AI systems. Fine-tunes models, evaluates performance, manages infrastructure, optimises inference, handles security, monitoring, and scalability.
Evidence: Builds and deploys AI solutions in production.
Evaluation: Discuss production experience by exploring topics such as deployment, model evaluation, latency, cost optimisation, security, monitoring, and reliability. They hould be able to explain technical decisions, lessons learned, and challenges from real-world AI systems they have built or maintained.
Level 5: AI Researcher / Innovator
Advances the field through research or significant innovation. Develops new architectures, training methods, evaluation techniques, or publishes original work.
Evidence: Conducts advanced AI engineering or research.
Evaluation: Evaluate original contributions rather than general knowledge. Discuss research publications, patents, open-source contributions, novel algorithms, benchmarking methodologies, or significant technical innovations. The focus should be on how the candidate has advanced AI beyond applying existing techniques.
A multifaceted professional known for diverse interests and contributions in various fields, including management, technology, education and entrepreneurship. Also the founder of ITPro.lk, bringing a unique perspective and industry knowledge to writing.