The open-versus-closed AI debate is usually framed as a belief system. One side argues that downloadable weights create sovereignty and lower costs. The other points to stronger frontier performance and a managed API that works in minutes.
A project does not need a belief. It needs a decision.
The durable way to choose is to score the workload across quality, total cost, latency, data sensitivity, customization, operational capacity, reliability, and switching risk. Model names change monthly; these constraints do not.
The 60-second decision tree
text
Must data or inference remain inside your environment?
├─ Yes → Can a deployable open-weight model meet your quality threshold?
│ ├─ Yes → Prefer open-weight/self-hosted; evaluate operations and license
│ └─ No → Use an approved private managed deployment or redesign the task
└─ No → Is the task narrow, repetitive, high-volume, or latency-sensitive?
├─ Yes → Compare self-hosted total cost with API cost at expected utilization
└─ No → Start with a managed closed API and benchmark alternatives
Need weight-level fine-tuning, offline use, or hardware control?
├─ Yes → Open-weight is the natural candidate
└─ No → Keep both classes in the bake-off
This is a starting filter, not a verdict. A model still has to pass your task evaluation.
First, use the terms correctly
Closed model usually means the provider exposes a hosted product or API but does not distribute weights. You buy access and accept its infrastructure, policies, update schedule, and pricing.
Open-weight model means you can obtain model weights and run them under a license. It does not guarantee open training data, source code, unrestricted commercial use, reproducibility, or the rights associated with conventional open-source software.
Open source AI is contested and license-specific. For procurement, skip the label and record exactly what is available: weights, code, data, architecture, license, modification rights, redistribution rights, and acceptable-use restrictions.
You have platform, security, and ML serving skills
You want the vendor to run infrastructure
Reliability
You can build redundancy and pin versions
A provider SLA and multi-region service are valuable
Portability
Weight ownership and deterministic versions matter
Rapid access to new proprietary capability matters
Do not average blindly. A hard regulatory boundary can outweigh seven moderate advantages.
1. Quality: test your task, not the leaderboard
A closed frontier model may lead difficult coding, multimodal reasoning, or long-horizon tool work. A compact open-weight model may be equally useful for classification, extraction, summarization, or a narrow domain workflow.
Build an evaluation set from real inputs. Include ordinary cases, long tails, unsafe requests, missing data, and tool failures. Measure accepted outcomes and human repair—not aesthetic preference.
If a closed model scores 94% and an open model 92%, the difference may or may not matter. For a reversible marketing draft, privacy or cost could dominate. For a medical extraction feeding care, two points may be unacceptable.
Learn to interrogate public scores with how to read an AI benchmark; then construct a private benchmark that reflects your loss function.
A “free” model on a $2-per-hour GPU costs $1,440 for a 30-day month before engineers, even if traffic uses only 15% of capacity. At high stable utilization, the same server can produce predictable economics. At low or variable demand, a per-token API often wins.
Closed APIs add network travel and shared-service variability. Providers compensate with optimized serving, large clusters, and regional infrastructure. Self-hosting places a small model near the application and can deliver consistent low latency or work offline.
Measure end-to-end P50 and P95 latency, including retrieval and tools. A local model generating slower tokens can still return first useful output sooner. A hosted model can beat an under-optimized local deployment.
Location also matters for factories, field devices, regulated environments, and products with intermittent connectivity.
4. Data sensitivity and governance
Self-hosting can keep prompts and outputs inside a controlled network. It does not automatically make the system safe. You still need access controls, secret handling, logging rules, patching, model-file provenance, and protection against prompt injection.
Closed providers offer enterprise retention controls, regional processing, encryption, contractual commitments, and compliance attestations. Those may meet policy more effectively than an improvised internal cluster.
Classify data before selecting a model:
Public
Internal
Confidential
Regulated or contract-restricted
Safety-critical
Then document where prompts, retrieved files, telemetry, backups, and human review travel. “The model is open” answers none of those questions by itself.
5. Fine-tuning and customization
Choose weight-level control when you need to alter behavior deeply, deploy an adapter locally, conduct model research, or control the serving stack. Open weights give the broadest technical freedom, subject to license.
Do not fine-tune merely because the model lacks current facts. Use retrieval-augmented generation for changing knowledge. Use prompting for instruction and output format. Use the prompting vs fine-tuning vs RAG decision tree before paying for training.
Closed providers may offer supervised fine-tuning without distributing weights. That can be ideal when customization matters but infrastructure ownership does not.
6. Operational capability
Running a model is easy. Running it reliably is a product.
You need capacity planning, batching, autoscaling, observability, model registries, vulnerability response, access controls, rollback, and performance tests. Quantized formats and inference engines evolve. Hardware failures and driver incompatibilities are ordinary work.
If one platform engineer spends a quarter supporting a system that saves $500 a month in API fees, the business case fails. If data sovereignty enables a major contract, the same work may be cheap.
7. Reliability and change control
A managed API can provide global capacity and an SLA, but it can also change model behavior, rate limits, safety policies, or availability. Pin versions where possible and test migrations.
Self-hosting lets you keep an exact artifact, but you own redundancy. A single workstation is not an SLA. Record recovery time, spare capacity, and fallback behavior.
A hybrid design can route to a second provider or local model. Avoid making the fallback so different that it silently violates quality or privacy requirements.
8. Lock-in and license risk
Closed lock-in appears in prompts tuned to a vendor, proprietary tool schemas, stored assistants, fine-tuning data formats, and unavailable features. Open-weight lock-in appears in GPU choices, inference engines, quantization formats, custom forks, and restrictive licenses.
Portability is not binary. Reduce it with an internal request schema, provider adapters, portable evals, exportable data, and a documented fallback. Test the fallback quarterly instead of admiring it in an architecture diagram.
Four common project decisions
Internal document assistant
Start with RAG on an enterprise API if contractual controls satisfy policy and the team lacks ML operations. Consider self-hosting when documents cannot leave the network, usage is steady, and a qualified smaller model passes citation tests.
High-volume extraction
Benchmark small open-weight models aggressively. Structured, repetitive inputs often justify a controlled local deployment. Keep a strong API fallback for ambiguous cases and measure the cascade.
Frontier research copilot
Closed frontier models often provide the fastest path to top reasoning and multimodal capability. Preserve source verification and provider portability. A local model may handle sensitive preprocessing.
Offline field application
Open-weight usually wins because connectivity is a hard constraint. Model size, quantization, device power, and update distribution dominate. See our laptop model guide for hardware reality.
A two-week bake-off
Select 50–200 representative tasks.
Define accept/reject rules before seeing outputs.
Test at least one closed and one open-weight candidate.
Record quality, repair, tokens, latency, and failure mode.
Model API cost and self-hosted cost at 20%, 50%, and 80% utilization.
Review data flow, license, security, and operational ownership.
Choose a primary and a tested fallback.
Re-run after meaningful model or price changes.
The final document should be a decision record, not a fan-club argument.
Bottom line
Choose closed when you need frontier capability quickly, demand is uncertain, and provider controls meet your requirements. Choose open-weight when deployment control, privacy, customization, predictable high-volume economics, locality, or version ownership creates enough value to fund operations.
Choose both when workload tiers differ. The best architecture is often a policy router: a controlled efficient model for routine work, a frontier API for genuinely hard exceptions, and evaluations deciding—not ideology.
Write the exit plan before signing
Whichever class wins, document how the project exits. For a closed API, preserve prompts, evaluation cases, source data, and an internal request schema that another provider can implement. Record which proprietary features have no equivalent and test one fallback on a small traffic slice.
For an open-weight deployment, preserve the exact weight hash, license, quantization artifact, inference engine, container, hardware assumptions, and rollback model. Confirm that the team can rebuild the service from records rather than from one engineer's workstation.
An exit plan does not require pretending providers are interchangeable. It exposes the real switching cost while leverage still exists. Review it when prices, model behavior, data terms, licenses, or hardware change. A project with a slightly weaker primary model and a working fallback can be more resilient than a benchmark leader tied to an undocumented stack. Portability is an operational practice, not a checkbox labeled “open.”
Model capabilities, prices, licenses, and provider controls change. This framework is intentionally model-agnostic; verify current documentation and run task-specific security, legal, quality, and cost reviews before deployment.
