Ask

Current answer

As of July 2026, the best local coding model depends on your hardware. For systems with 16 GB RAM, quantised Llama 4 (8B) and DeepSeek Coder V3 (7B) lead community benchmarks. For 32 GB+, Qwen 3 Coder (32B) at 4-bit quantisation currently holds the top spot in independent tests.

Why

These rankings are based on independent community benchmarks on HumanEval, SWE-bench Verified, and MultiPL-E across multiple hardware configurations, not vendor-reported scores.

Supporting evidence

• Community benchmark aggregator (July 2026)
• Independent hardware-specific tests
• SWE-bench Verified leaderboard

Points of disagreement

Some users report that CodeLlama 4 (13B) performs better on specific languages like Rust and Go. The ranking varies by programming language.

Practical recommendation

If you have 16 GB RAM, start with Llama 4 (8B) at Q4_K_M quantisation via Ollama or LM Studio.

Current answer

For 16 GB RAM systems, the strongest overall performers are Llama 4 (8B) at Q4_K_M, DeepSeek Coder V3 (7B), and Phi-4 (14B) at Q3_K_M. These all run comfortably within 16 GB while leaving room for context.

Why

These recommendations are based on real-world testing across common quantisation formats (GGUF) and inference engines (Ollama, LM Studio, llama.cpp).

Supporting evidence

• r/LocalLLaMA community benchmarks
• Independent hardware tests (July 2026)
• Ollama model library

Points of disagreement

Phi-4 at heavy quantisation (Q3) loses some reasoning quality. Some users prefer the smaller but less quantised Llama 4 (8B) at Q5.

Practical recommendation

Start with Llama 4 (8B) Q4_K_M for general use. Add DeepSeek Coder V3 for coding-specific tasks.

Current answer

As of July 2026, for frontier models, Anthropic (Claude 4 Haiku) leads price-to-speed. For mid-tier, Groq and Together AI offer the best latency-per-dollar. For open-weight model hosting, Fireworks AI and Together AI are most competitive.

Why

Comparison is based on published API pricing, independent latency measurements, and throughput benchmarks across equivalent model tiers.

Supporting evidence

• Provider pricing pages (July 2026)
• Independent API latency benchmarks
• artificialanalysis.ai comparison data

Points of disagreement

Groq offers superior latency but limited model selection. Together AI offers more models at slightly higher latency.

Practical recommendation

Use Anthropic for frontier quality, Groq for maximum speed, and Together AI for the widest open-weight model selection.

Current answer

The current consensus (July 2026) is moving away from monolithic vector databases toward hybrid approaches: structured working memory (key-value + relational) for active context, semantic retrieval for long-term reference, and episodic summarisation for conversation history. MemGPT-style hierarchical memory and Mem0-style persistent user memory are the leading patterns.

Why

This represents convergence across multiple research groups (Berkeley, Stanford, Microsoft) and production systems (LangChain, LlamaIndex, CrewAI).

Supporting evidence

• MemGPT paper (Berkeley, 2024)
• Mem0 architecture documentation
• LangChain Memory module docs
• Letta (MemGPT successor) v0.5 release notes

Points of disagreement

Debate continues on whether agent memory should be model-managed (MemGPT approach) or framework-managed (LangChain approach). The hybrid consensus is emerging but not settled.

Practical recommendation

For new projects, start with a simple key-value store for working memory and add semantic retrieval only when needed. Evaluate Mem0 or Letta for persistent user memory.

Current answer

GPT-5 was released on 10 July 2026. As of 12 July 2026, independent benchmark verification is still in progress. Vendor-reported scores on coding and reasoning benchmarks are available but should be treated as provisional until independent results are published.

Why

Early community testing is underway. Full independent verification typically takes 1–3 weeks after a major release.

Supporting evidence

• OpenAI GPT-5 technical report (10 July 2026)
• Early community reports on r/MachineLearning
• SWE-bench submission pending

Points of disagreement

Early community reports are mixed on real-world coding performance versus vendor-reported benchmark scores.

Practical recommendation

Wait for independent SWE-bench and HumanEval+ results before making procurement decisions. Expect initial independent results by 17–24 July.

Current answer

July 2026 saw three significant MCP gateway developments: (1) Anthropic released MCP Gateway v2 with streaming tool results and improved authentication, (2) the open-source mcpx gateway added dynamic tool discovery, and (3) Cloudflare announced an MCP gateway product integrated with Workers.

Why

MCP gateway competition is accelerating as tool-use becomes standard in agent workflows. Streaming results and dynamic discovery are the key differentiators.

Supporting evidence

• Anthropic MCP Gateway v2 changelog
• mcpx v0.9 release notes
• Cloudflare blog (July 2026)

Points of disagreement

Whether to use provider-specific gateways (Anthropic) or provider-agnostic (mcpx/Cloudflare) depends on your model mix.

Practical recommendation

If you use primarily Anthropic models, MCP Gateway v2 is the path of least resistance. For multi-provider setups, evaluate mcpx or Cloudflare.

Current answer

As of July 2026, the most reliably measured coding agents on SWE-bench Verified are: Devin (Cognition) at 38.7%, Claude Code (Anthropic) at 35.2%, and the open-source SWE-Agent at 29.1%. However, 'reliability' also depends on cost, latency, and task type.

Why

SWE-bench Verified is currently the best available measure of real-world coding task completion, though it has limitations.

Supporting evidence

• SWE-bench Verified leaderboard (July 2026)
• Independent agent comparison reports
• Community experience reports

Points of disagreement

SWE-bench scores don't capture iteration speed, cost, or developer experience. Some developers report better day-to-day reliability from Cursor + Claude despite lower benchmark scores.

Practical recommendation

For autonomous bug-fixing, Devin leads. For interactive coding, Claude Code or Cursor + Claude. For budget-conscious teams, SWE-Agent with Llama 4 is viable.

Current answer

This depends on the specific tool. Always check: (1) the exact licence text, not just the label, (2) whether all dependencies are also openly licensed, (3) whether the model weights or only the inference code is open, (4) whether there are commercial-use restrictions. Many 'open-source' AI tools use custom restricted licences that are not OSI-approved.

Why

The term 'open source' is frequently misused in AI. OSI has published an Open Source AI definition, but adoption is inconsistent.

Supporting evidence

• OSI Open Source AI Definition v1.0
• Model licence registry projects
• Software Freedom Conservancy AI analysis

Points of disagreement

The definition of 'open source AI' itself is contested between OSI, FSF, and industry groups.

Practical recommendation

Use our licence classification: check the specific licence, commercial-use terms, and data/model/code openness separately. Don't rely on marketing labels.

Current answer

Current AI security best practices (July 2026) centre on: (1) prompt injection prevention via input sanitisation and structured prompting, (2) output validation and sandboxing for tool-using agents, (3) model access control and rate limiting, (4) data isolation between tenants, (5) adversarial input detection, and (6) regular red-teaming including indirect prompt injection scenarios.

Why

OWASP Top 10 for LLM Applications v2.0 and NIST AI 600-1 provide the current authoritative frameworks.

Supporting evidence

• OWASP Top 10 for LLM Applications v2.0
• NIST AI 600-1
• ANSSI AI security recommendations
• NCSC AI security guidance

Points of disagreement

Debate continues on whether RAG-based systems need fundamentally different security models from traditional web applications.

Practical recommendation

Start with the OWASP LLM Top 10. Implement structured prompting (XML/JSON schemas) to reduce injection surface. Sandbox all tool executions.

Current answer

As of July 2026, Llama 4 (405B) is the strongest open-weight model for commercial use, followed by Qwen 3 (72B) and DeepSeek V3 (671B MoE). All three have commercially permissive licences. For smaller deployments, Llama 4 (70B) offers the best performance-per-parameter ratio.

Why

Independent benchmarks across MMLU, HumanEval, GSM8K, and HellaSwag confirm these rankings. Licence terms have been verified.

Supporting evidence

• Open LLM Leaderboard (July 2026)
• Independent model comparisons
• Official licence documents

Points of disagreement

DeepSeek V3's mixture-of-experts architecture means 671B total parameters but only ~37B active per token, making comparisons with dense models complex.

Practical recommendation

For maximum capability, Llama 4 (405B). For cost efficiency, DeepSeek V3. For balance, Qwen 3 (72B). All three are commercially usable.

Current answer

The most cost-effective local AI setup (July 2026) is a used RTX 3090 (24 GB VRAM, ~£550 used) running Ollama or LM Studio. This comfortably runs Llama 4 (8B) at Q8, Llama 4 (70B) at Q3, or DeepSeek Coder V3 at Q6. For CPU-only, a Mac Mini M4 (24 GB unified memory, £599) runs 7B–14B models at acceptable speeds.

Why

Price-to-performance analysis based on current used hardware prices and inference benchmarks.

Supporting evidence

• r/LocalLLaMA hardware threads (July 2026)
• Independent inference benchmarks
• Current used hardware prices (eBay, CeX)

Points of disagreement

Apple Silicon offers better power efficiency but less software flexibility. NVIDIA offers CUDA compatibility and wider model support.

Practical recommendation

RTX 3090 used for maximum capability on a budget. Mac Mini M4 for quiet, efficient, CPU-based inference. Avoid new RTX 4090s unless you need the absolute fastest token generation.

Current answer

For structured JSON/output generation (July 2026): GPT-5 with structured outputs mode leads, followed by Claude 4 with tool-use JSON mode, and Llama 4 (70B) with outlines/guidance libraries. For local structured output, Llama 4 (8B) with llama.cpp grammars performs well.

Why

Structured output reliability is measured by schema adherence rate, not just output quality. GPT-5 currently achieves >99% schema adherence.

Supporting evidence

• Provider structured-output documentation
• Independent schema-adherence tests
• Community comparison threads

Points of disagreement

For simple schemas, open-weight models with grammars achieve comparable adherence. The gap widens for complex, nested schemas.

Practical recommendation

GPT-5 for complex structured output in production. Llama 4 + grammars for local/private structured output. Claude 4 for balanced quality and schema support.

Current answer

For startups (July 2026), the best API depends on your primary need: Anthropic (Claude 4 Haiku) for cost-effective coding and reasoning, OpenAI (GPT-5) for maximum capability and ecosystem maturity, and Groq for latency-sensitive applications. Together AI offers the best value for open-weight model hosting.

Why

Comparison factors: pricing, latency, reliability, model selection, SDK maturity, and startup programme benefits.

Supporting evidence

• Provider pricing pages
• API latency benchmarks
• Startup programme comparisons (Anthropic, OpenAI, Google)

Points of disagreement

Startups in the EU may prefer Anthropic or Mistral for data-residency reasons. Google Cloud offers the best startup credits programme ($200K+).

Practical recommendation

Start with Anthropic Claude 4 Haiku for cost efficiency. Add GPT-5 for tasks requiring maximum capability. Apply for startup credits from all major providers.

Current answer

As of July 2026, Gemini 3 Pro (2M token context) leads raw context capacity. Claude 4 (200K) leads in-context retrieval accuracy (needle-in-haystack >99%). GPT-5 (256K) offers the best balance of context length and reasoning quality at long ranges.

Why

Long-context performance degrades differently across models. Raw context size ≠ effective context utilisation.

Supporting evidence

• RULER benchmark (July 2026)
• Needle-in-haystack tests
• Provider documentation

Points of disagreement

For most practical use cases, 128K–200K tokens are sufficient. The 2M token ceiling of Gemini is rarely needed outside document-processing pipelines.

Practical recommendation

Claude 4 for tasks requiring precise retrieval from long documents. GPT-5 for long-form reasoning. Gemini 3 Pro for extreme context needs (full codebase analysis).

Current answer

Model routers (July 2026) are increasingly viable. OpenRouter, Martian, and Portkey offer routing across 200+ models. The value proposition is real: cost savings of 30–60% by routing simple queries to smaller models while reserving frontier models for complex tasks.

Why

Router quality has improved significantly. Modern routers use lightweight classifiers to predict task difficulty before selecting a model.

Supporting evidence

• OpenRouter docs and pricing
• Martian Model Router benchmarks
• Independent router comparison (July 2026)

Points of disagreement

Some developers prefer explicit model selection for predictability. Routers add a layer of opacity to model choice.

Practical recommendation

Use a router for cost-sensitive applications with varied query difficulty. For latency-critical or compliance-sensitive applications, select models explicitly.

Current answer

Prompt engineering best practices have evolved (July 2026): (1) use structured prompts (XML or JSON schema) rather than prose for complex tasks, (2) provide 3–5 diverse examples rather than 1, (3) specify output format explicitly, (4) use chain-of-thought only for reasoning tasks where it demonstrably improves accuracy, (5) separate system instructions from user input clearly, and (6) test prompts against adversarial inputs.

Why

These practices are informed by the latest research on prompting effectiveness, including the 'structured prompting beats clever prompting' consensus.

Supporting evidence

• Anthropic prompt engineering guide (2026)
• OpenAI prompt engineering guide
• Research: 'Structured Prompts Outperform Creative Prompts' (June 2026)

Points of disagreement

Chain-of-thought prompting can increase latency and cost without improving accuracy for simple tasks. Use it selectively.

Practical recommendation

Start with structured XML prompts. Add examples for ambiguous tasks. Use chain-of-thought only when accuracy matters more than speed.

Current answer

To evaluate benchmark trustworthiness, check: (1) Who ran it — vendor or independent? (2) Is the code and data public? (3) What prompting method was used? (4) Was the benchmark contaminated in training data? (5) Is the benchmark saturated? (6) Were multiple runs averaged? (7) Were the settings comparable across models? Use our benchmark health labels as a starting point.

Why

Benchmark results are frequently incomparable due to different prompting methods, hardware, sampling settings, or contamination.

Supporting evidence

• The Trace Manifest benchmark methodology
• Research on benchmark contamination
• Independent evaluation standards

Points of disagreement

Some argue that 'contamination' is inevitable and not always harmful. Others argue it fundamentally invalidates benchmark results.

Practical recommendation

Trust independent, reproducible benchmarks over vendor-reported results. Check our benchmark registry for health status before citing scores.

Current answer

Open-weight: model weights are publicly downloadable but training data and code may be private (e.g., Llama 4). Open-source (OSI definition): weights, training code, data, and methodology are all publicly available under an OSI-approved licence (very rare for frontier models). Source-available: code is public but under restrictive terms (e.g., no commercial use). Most 'open-source' AI models are actually open-weight.

Why

These distinctions matter for commercial use, reproducibility, and licence compliance.

Supporting evidence

• OSI Open Source AI Definition v1.0
• Model licence analysis
• Software Freedom Conservancy guidance

Points of disagreement

Meta and others contest the OSI definition, arguing that open-weight alone should qualify as 'open source'. The OSI and FSF disagree.

Practical recommendation

Always check the specific licence terms. 'Open-weight' is the safest accurate term for most downloadable models. Reserve 'open-source' for OSI-compliant releases.

Current answer

The current best local inference tools (July 2026): Ollama for ease of use and model management, LM Studio for GUI-based discovery and testing, llama.cpp for maximum performance and flexibility, vLLM for production serving, and text-generation-webui for experimentation. For Apple Silicon, MLX (Apple's framework) offers the best performance.

Why

Tool choice depends on your use case: experimentation vs production, GUI vs CLI, single-user vs multi-user serving.

Supporting evidence

• Official tool documentation
• Community performance benchmarks
• r/LocalLLaMA tool comparison threads

Points of disagreement

Ollama is easiest but abstracts away configuration. llama.cpp offers more control but requires more expertise.

Practical recommendation

Start with Ollama for personal use. Move to LM Studio if you prefer a GUI. Use vLLM for production serving. Use MLX on Apple Silicon.

Current answer

Yes. The EU AI Act (enforced from August 2026) classifies AI systems by risk tier. Most developer tools and general-purpose models fall under limited or minimal risk with transparency obligations. High-risk applications (healthcare, hiring, law enforcement) have stricter requirements. The US has executive orders but no comprehensive federal law yet. The UK has a sectoral approach. Check your jurisdiction.

Why

Regulatory requirements vary by region, use case, and model classification.

Supporting evidence

• EU AI Act final text
• UK AI regulation white paper
• US Executive Order on AI
• Provider compliance documentation

Points of disagreement

The EU AI Act's classification of general-purpose AI is still being interpreted. Guidance is evolving.

Practical recommendation

For EU deployments, review your AI Act tier classification. Most SaaS and developer-tool use cases are limited risk. Consult legal counsel for high-risk applications.